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ABSTRACT 


Analytic  solutions  are  sought  for  various  special  cases  of 
a  general  scheduling  model  based  on  the  following  assumptions: 

(1)  work  is  scheduled  on  a  single  processor,  (2)  the  scheduling 
objective  is  to  maximize  service  rendered,  C3)  a  loss  function 
associated  with  each  request  reflects  the  decline  in  utility 
caused  by  its  delayed  completion,  C4)  each  request  is  associated 
with  a  request  class  for  which  the  distribution  of  service  times 
is  known,  (5)  the  process  by  which  new  requests  arrive  is  known 
for  each  request  class,  and  (.6)  preemption  of  a  request  not  yet 
completed  requires  a  fixed  amount  of  processor  time.  After  a 
detailed  examination  of  the  model  and  a  discussion  of  past  work 
on  related  models,  various  special  cases  are  studied.  First,  free 
preemption,  linear  losses,  and  no  arrivals  are  assumed.  A  new 
scheduling  discipline  called  "Smallest  Rank",  or  SR,  is  defined, 
and  its  optimality  is  proven.  Next,  arrivals  are  considered. 

With  free  preemption,  linear  losses,  known  service  times,  and 
Poisson  arrivals,  a  well  known  rule,  SRPT/c,  is  shown  to  be 
optimal  within  a  broad  class  of  practical  scheduling  rules.  An 
argument  is  made  that  the  class  necessarily  encompasses  the 
globally  optimal  rule.  Finally,  significant  preemption  costs  are 
considered.  An  optimal  preemption  strategy  is  derived  and  proven 
for  a  simple  system.  Consideration  is  also  given  to  a  system 
with  a  hyperexponential  service  time  distribution,  a  distribution 
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found  empirically  in  many  time-sharing  computer  systems.  Some 
other  cases  are  discussed  briefly,  then  generalizations  of  the 
model  suitable  for  future  study  are  proposed. 
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CHAPTER  I 


AN  ANALYTIC  SCHEDULING  MODEL 


The  Source  of  the  problem 

An  important  advance  in  computer  system  design  was  the 
development  of  a  method  by  which,  at  reasonable  expenditure  of 
time  and  resources,  the  processing  of  a  request  could  be 
suspended  for  a  time,  then  later  be  resumed  from  the  point  of 
interruption.  This  advance  allows  not  only  better  component 
utilization,  increased  thruput,  and  lower  computation  costs  by 
the  ability  to  overlap  activities  of  various  system  components, 
but  also  it  allows  the  allocation  of  system  resources  to  be 
continually  reconsidered.  Thus  the  presumption  of  traditional 
scheduling  strategies  that  once  processing  of  a  request  is  initi¬ 
ated,  it  must  be  processed  to  completion  is  no  longer  true,  and 
development  of  more  general  scheduling  strategies  is  necessary. 

The  ability  to  preempt  the  processing  of  one  request  in 
favor  of  another  at  any  time  has  created  a  scheduling  environment 
of  such  diverse  possibilities  that  it  is  still  not  fully  under¬ 
stood.  Even  in  the  simplest  case  where  a  processor  is  the  single 
critical  resource  with  respect  to  scheduling,  the  comparative 
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evaluation  of  various  scheduling  strategies  is  a  complex  problem. 

A  principle  aim  of  this  research  is  the  determination  of  optimal 
strategies  for  scheduling  a  single  processor.  The  attainment  of 
this  goal  is  of  theoretic  merit,  but  with  regard  to  applicability, 
its  importance  will  be  greatest  as  a  tool  in  similar  work  dealing 
with  the  simultaneous  allocation  of  several  resources . 

The  Model  to  be  investigated 

The  analytic  model  chosen  for  investigation  in  this  research 
is  based  on  six  assumptions.  In  this  section,  definitions  and 
notation  will  be  presented  along  with  the  assumptions.  In  order 
to  distinguish  this  model  from  others,  we  shall  call  it  MODEL Q. 

Assumption  1. — The  processor  is  the  single  scarce  resource 
to  be  scheduled. 

There  is  a  sequence  of  requests  numbered  1,  2,  ...  sequen¬ 

tially  upon  their  arrival  to  the  system.  A  function  £ (x) ,  with  a 
domain  of  the  nonnegative  reals  and  a  range  of  the  nonnegative 
integers  is  called  the  processor  assignment  function.  If  E(x)=n 
for  n>0,  we  say  request  n  is  being  served  or  the  processor  is 
assigned  to  request  n  at  time  x.  If  l(x)=0,  no  request  is  being 
served  at  time  x. 

Assumption  2. — Each  request  may  be  associated  with  a  request 
class  for  which  the  distribution  of  service  times  is  known. 

The  jth  request  is  described  by  its  arrival  time,  a j ,  its 
actual  service  time,  t j ,  and  its  class  identity  number,  Cj .  The 
class  identity  of  a  request  is  determined  by  characteristics  of 
the  request  which  are  known  upon  its  arrival.  The  actual  service 
time  of  a  request  is  not  necessarily  known  to  the  system  until 
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service  to  the  request  is  completed. 

The  cumulative  distribution  of  service  times  for  class  i 

requests  is  known  and  is  denoted  by 

F. (x)  =  Prob[t.<x  I  c.=i] . 

1  D  1  3 

When  F^ (x)  is  differentiable,  the  density  function  of  service 
times  is 


f±(x) 


d_ 

dx 


Fi (x)  . 


We  associate  with  each  request,  j,  a  request  assignment  function. 


a j (x) .  Its  value  is  defined  by 


CXj  (x)  = 


f  1  1 

(  0  o 


f  E (x) = j  and  x>a . 

j 

otherwise . 


Note  that  because  E  (x)  is  single-valued, 


l  Oa  Cx)  <  1  Vx  • 


j=l 

Intuitively,  this  means  that  at  most  one  request  can  be  assigned 
to  the  processor  at  a  time.  The  attained  service  of  request  j  at 
time  x  is  defined  as 


b j  (x)  =  min 


[v  /a  aj(Y)dy]- 


The  completion  time  of  request  j  is  ej  =  min[x  |  b_.(x)=t^].  The 


set 

of  requests  in  system 

at  time 

X 

is  J(x)  =  {j 

a  .  <xAb  .  (x) <t 

The 

time  in  queue  without 

service 

of 

request  j  at 

3  3 

time  x  is 

q  .  (x)  = 

min  x- 

a  •  v 

r  x 

max  y  1  / 

a  .  (s)  ds  =  oil 

3 

L 

3 

l 

x-y 

3  J  J 

The  constraint  E (x)  =  j  only  if  j  e  J (x)  has  been  avoided 
in  these  definitions.  However 


E  (x)  =  j,  j  /  J(x)  ==> 


db .  (t) 

l 

dt 


t=x  =  o\/i- 
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Thus  no  request  is  progressing  toward  completion,  and  such  an 
assignment  generally  is  undesirable. 

With  attained  service  y,  request  j  of  class  i  has  a  known 
conditional  cumulative  remaining  service  time  distribution  given 
by 

GiCx,y)  =  Prob  [t_.-y<x  |  c^=i  a  t^>y]  , 

Fi(x+y)  -  FiCy) 

=  l-Fi(y)  . 

If  (x)  is  differentiable,  the  density  function  of  remaining 
service  time  is 

g.  (x,y)  =  iL_G  .  (x ,y) 

1  9x  2 3- 

f ±  Cx+y) 

=  1  -  F± (y)  * 

The  expected  service  time  of  request  j  of  class  i  is 

CO 

E [t .  |  c  .=i]  =  /  x  dF  .  (x)  . 

J  3  0 *  1 

With  attained  service  y,  the  expected  remaining  service  time  of 
request  j  of  class  i  is 

00 

E[t.-y  |  c  •  =i  a  t .  >y]  =  /  x  9  G-(x,y). 

3  J  J  0 

Assumption  3 . — The  value  of  completing  a  request  is  a 
non-increasing  function  of  its  time  of  completion  for  any  fixed 
arrival  time. 

With  each  request  class,  i,  we  associate  a  loss  function, 
l^(t) ,  which  indicates  the  decrease  in  value  of  service  rendered 
between  instantaneous  completion  and  completion  t  time  units  after 
arrival.  Loss  functions  are  monotone  non-decreasing  as  long  as 
information  created  in  servicing  a  request  can  be  preserved  at 


5 


negligible  cost.  An  important  special  case  occurs  when 

^_1- (t)  =  k.,  a  constant.  This  is  the  case  of  linear  loss 
at  x  i  - 

functions .  The  constant  is  the  linear  loss  rate  of  class  i 

requests.  If  k^  =  k^  for  all  request  classes,  i  and  j,  we  have 

identical  linear  losses. 

Assumption  4. — Using  some  processor  time,  an  unfinished 
request  may  be  removed  from  the  processor  and  later  be  resumed 
from  the  point  of  interruption. 

Let  C  denote  the  amount  of  processor  time  lost  in  the 
overhead  of  preempting  one  request  in  favor  of  another.  Then  the 
processor  assignment  function  is  bound  by  the  constraint 

(hioE(x+h)  *  ECx>  =  j  >  o)  A  Cbj(x)  <  V 

==>  E(s)  =  0,  x<s<x+C. 

That  is,  for  C  units  after  the  preemption  of  an  unfinished 
request,  the  processor  cannot  be  assigned  to  any  other  request. 
When  preemption  overhead  is  presumed  negligible,  we  have  C  =  0. 

Assumption  5. — The  process  by  which  new  request  arrivals 
will  occur  is  known. 

We  assume  the  arrival  process  to  be  dependent  on  at  most 
the  current  time  and  the  arrival  times  and  classes  of  all  previous 
arrivals  to  the  system.  That  is,  the  arrival  process , 

A(t,  St,  £,  i  ,x)  ,  expresses  the  probability  at  time  t  that  the  next 
arrival  will  be  of  class  i  and  will  occur  by  t+x  if  a  and  c  are 
the  vectors  of  arrival  times  and  class  identities  of  past  arrivals 
A(t,a,c,i,x)  =  Prob[cn+]_=i  a  an+^<t+x 

{ a  ^ , . . . , an } , 

^={c;l '  •  •  •  'Cn^  * 
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In  most  cases,  arrivals  of  one  class  are  independent  of 
arrivals  of  other  classes.  Then  the  arrival  process  for  class  i 
requests  may  be  expressed  as  a  function  of  only  the  current  time 
and  the  previous  arrival  times  of  class  i  requests.  A  stationary 
arrival  process  does  not  depend  on  the  current  time,  but  only  on 
the  time  intervals  since  previous  arrivals  of  the  same  class. 
General  independent  arrivals  occur  when  the  arrival  process 
depends  only  on  the  time  since  the  last  arrival  of  the  same  class. 
In  this  situation,  interarrival  times  are  independent  samples 
from  some  distribution.  Poisson  distributed  or  random  arrivals 
arise  when  interarrival  times  are  exponentially  distributed. 

Then 

.  \  i  — 1 1  x 

A  (t ,  a ,  c ,  l ,  x)  =  1  -  e 

where  A^  is  the  parameter  of  the  exponential  distribution  of 
interarrival  times  of  class  i  requests.  Note  that  a  Poisson 
arrival  process  is  stationary  and  is  independent  of  all  previous 
arrival  times.  The  rate  of  arrivals  of  class  i  requests  is  A^. 

In  the  finite  horizon  or  no  arrivals  case, 

A(t,t,3,i,x)  =  0 

In  this  situation,  all  requests  to  be  scheduled  are  presumed  to 
be  available  for  service  from  time  zero. 

Assumption  6. — The  goal  of  scheduling  is  to  maximize  the 
total  value  of  service  rendered. 

A  scheduling  rule,  S ,  is  an  algorithm  for  specifying  the 
processor  assignment  at  time  t,  Eg  (t)  ,  as  a  function  of  the  set 
of  requests  in  system,  J (t) ,  and  the  arrival  process, 

A (t , a ,c , i ,x) .  Note  that  J (t)  depends  on  Eg (y) ,  0<y<t.  The 
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scheduling  rule  may  also  depend  on  the  current  time,  although  a 
stationary  arrival  process  and  a  scheduling  objective  which  is 
independent  of  time  make  it  sufficient  to  consider  only  stationary 
scheduling  rules  in  which 

J(t]_)  =  J(t2)  a  X(ti,ai,c1,i/x)  =  ACt2  ,a2  ,c2  ,i  ,x) 

==>  ZS  ^tl)  =  ZS (t2)  • 

It  is  important  to  notice  that  I (x) ,  and  in  turn  a j (x) , 

b j (x)  ,  ej ,  and  qj  (x)  all  depend  on  both  the  scheduling  rule  used 

and  the  particular  sequence  of  arrivals  encountered.  To  evaluate 

a  specific  scheduling  rule,  we  will  consider  all  possible  arrival 

patterns  and  consider  the  distribution  of  the  random  variables, 

e_;  .  Let  Ec[e^]  denote  the  expected  completion  time  of  request  j 
J  ^  J  - 

under  scheduling  rule  S. 

If  at  time  x,  scheduling  rule  S  assigns  request  j  for 
quantum  q,  then 

(ak<x  v  x+q<ak)  \J  k  ==>  Z (s)  =  j, 

x<s<x+min [q , t j-b j (x) ] . 

That  is,  unless  an  arrival  occurs,  the  processor  will  be  assigned 
to  request  j  for  the  next  q  time  units  unless  it  completes  before 
then.  If  j  is  assigned  to  run  to  completion  at  time  x,  then  it 
is  assigned  for  an  infinite  quantum. 

The  expected  rate  of  total  loss  over  all  requests  under 

scheduling  rule  S  is 

-  lim  1  £  _  r  i  x 

LS  ~  n+°°  n  .  ^  ^Cj  ^es  j aj)  • 
j=l  J  J 

Under  finite  horizon,  we  need  consider  only  the  expected  total 


8 


loss  under  scheduling  rule  S, 


n 


Because  loss  functions  reflect  declines  in  the  value  of  service 
rendered,  service  value  is  maximized  when  losses  are  minimized. 
Therefore,  our  objective  will  be  to  determine  scheduling  rules 
which  minimize  expected  rate  of  total  loss  or  expected  total  loss. 
Scheduling  rule  S  which  obeys  the  constraints  of  a  particular 
situation  is  the  optimal  scheduling  rule  for  that  situation  if 
and  only  if  Lg  <  Lg  ,  for  all  other  scheduling  rules,  S',  which 
also  obey  the  constraints  of  the  situation. 

The  Relevance  of  the  model 

Let  us  examine  the  justifiability  of  the  assumptions  in 
MODEL Q  with  respect  to  scheduling  problems  in  actual  computer 
systems.  The  general  scheduling  problem  in  current  computer 
systems  involves  allocation  of  many  resource  types  with  various 
constraints  on  ordering  and  simultaneity  of  resource  needs. 
However,  full  understanding  of  the  single  resource  scheduling 
problem  is  a  prerequisite  to  understanding  multiple  resource 
scheduling.  While  no  real  computer  systems  have  only  a  single 
resource,  some  systems  allocate  all  critical  resources  as  a  pack¬ 
age.  This  holds  true  when  only  a  single  program  may  reside  in 
memory  at  a  time.  In  other  systems,  the  processor  is  the  only 
scarce  resource  so  that  its  utilization  is  of  primary  importance. 
Frequently  in  on-line  systems  memory  requirements  are  small  and 
easily  met,  but  the  processor  must  be  contended  for.  The  current 
trend  of  decreasing  cost  of  memory  and  peripherals  relative  to 
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processors  makes  such  situations  increasingly  likely  in  future 
systems . 

If  computer  system  users  had  no  preferences  about  when  their 
processing  requests  were  completed,  there  would  be  no  scheduling 
problem.  The  problem  arises  because  many  factors,  most  of  which 
are  quite  subjective,  cause  completion  of  a  request  to  be  of  more 
value  to  a  user  sooner  than  later.  The  simple  assumption  that 
the  cost  of  saving  results  in  some  form  is  negligible  makes  the 
value  of  service  rendered  the  user  in  fulfilling  a  request  a 
monotone  non-increasing  function  of  its  completion  time.  Thus 
the  corresponding  loss  functions  are  monotone  non-decreasing. 

Their  precise  shape,  however,  is  influenced  by  many  factors,  some 
totally  unrelated  to  the  request  for  service  itself.  Some  of 
these  factors  are  decline  in  utility  of  results  (such  as  weather 
forecasts) ,  lost  opportunity  costs  (such  as  an  accurate  stock 
market  prediction),  frustration  costs,  alternative  tasks  on  the 
agenda  of  the  request  submitter,  and  many  others.  Such  judgments 
cannot  be  made  usefully  by  the  system,  and  are  at  best  roughly 
estimable  by  the  person  submitting  the  request.  Current  systems 
allow  the  user  an  equally  rough  means  of  expressing  his  loss 
function  to  the  system.  Most  common  is  the  expression  of  a  rela¬ 
tive  priority  number.  A  program  with  a  higher  relative  priority 
chan  another  is  presumed  to  have  a  more  steeply  increasing  loss 
function.  An  alternative  offered  in  some  systems  is  "overnight 
priority."  This  indicates  a  relatively  flat  loss  function  over 
the  period  of  a  least  8  to  12  hours.  A  somewhat  more  sophisti¬ 
cated  mechanism  allows  specification  of  a  due  time,  which  the 
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system  attempts  to  satisfy.  Such  specifications  are  normally 
closely  related  to  the  charging  scheme.  Higher  priority  costs 
more,  overnight  priority  is  inexpensive,  and  charging  is  propor¬ 
tional  to  the  difficulty  of  meeting  a  due  time. 

Two  points  of  a  loss  function  are  more  easily  determined 
than  the  rest  of  the  curve.  By  definition,  with  instantaneous 
completion,  loss  due  to  delay  is  zero  Clj(0)=0).  And,  after  some 
interval,  T j ,  the  user  is  nearly  indifferent  as  to  whether  he 
ever  obtains  the  results  (lj(Tj)  =*  lj  (°°)  )  .  Estimation  of  the 
general  character  of  loss  functions  between  t=0  and  t=Tj  is  an 
important  consideration  in  evaluating  scheduling  strategies. 
Figure  1  shows  some  possible  loss  functions  in  various  special 
situations.  Figure  1(A)  illustrates  a  situation  that  might  be 
encountered  in  a  real-time  control  system.  Computation  results 
are  needed  at  a  particular  time,  and  are  valueless  thereafter. 
Figure  1(B)  reflects  a  typical  loss  function  of  a  user  who  has 
submitted  his  request  (probably  through  a  batch  stream)  and 
returns  occasionally  to  see  if  it  is  done.  He  is  indifferent  to 
the  completion  time  between  his  sampling  points,  thus  creating  a 
step  function  of  loss.  Figure  1(C)  illustrates  possible  situa¬ 
tions  in  which  the  user  is  interacting  with  the  system  through  an 
on-line  terminal.  The  request  here  consists  of  a  single  inter¬ 
action  rather  than  an  entire  program.  Note  that  the  point  of 
total  indifference  here  is  reached  typically  in  time  on  the  order 
of  minutes  rather  than  on  the  order  of  hours  (and  possibly  days) 
as  in  the  situation  of  Figure  1(B).  At  a  single  interaction,  the 
user's  personality  and  dedication  determine  whether  he  will  react 
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Figure  1. — Loss  function  forms. 
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with  growing  impatience  Cc^)  ,  growing  apathy  Lc^\  ,  or  just  con¬ 
stant  consternation  (cii:i_)  .  This  reaction  might  vary  signifi¬ 
cantly  from'  one  interaction  to  the  next.  Figure  1(D)  is  a  form 
of  loss  function  which  may  be  used  to  approximate  all  loss  func¬ 
tions.  The  justifications  are: 

1.  In  the  situation  of  Figure  1(B) ,  the  sampling  points 
at  which  the  user  will  seek  his  request  are  generally 
unpredictable  in  advance.  Thus  the  expected  loss 
function  which  results  from  anticipating  various 
future  sampling  patterns  is  continuous  and  generally 
linear  in  its  rise. 

2.  In  the  situation  of  Figure  1(C),  we  might  anticipate 
that  over  a  long  sequence  of  interactions,  frustra¬ 
tion  costs  are  generally  outweighed  by  the  cost  of 
project  delay.  Then  the  principal  loss  incurred  is 
proportional  to  the  total  time  spent  at  the  console 
from  the  start  of  program  development  to  the  com¬ 
pletion  of  the  particular  goal.  This  would  allow 
linear  approximation  of  the  loss  functions  at 
individual  interactions. 

Figure  1(E)  shows  the  ultimate  simplification  of  the  loss  function 
to  a  pure  linear  form.  A  typical  general-purpose  computing  facil¬ 
ity  in  normal  operation  seldom  must  turn  customers  away  because 
it  cannot  service  them  before  their  point  of  indifference.  This 
indicates  that  service  completion  normally  occurs  in  the  linearly 
increasing  portion  of  the  bounded-linear  loss  function.  At  worst 
a  pre-screen  mechanism  may  be  added  to  a  system  for  the  purpose  of 
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allowing  through  only  a  mixture  of  requests  all  of  which  can  be 
completed  with  positive  utility.  In  these  cases,  the  system  may 
ignore  the  existance  of  points  of  total  indifference  without 
catastrophic  consequences,  and  assume  loss  functions  to  be  linear. 
In  this  research,  for  the  most  part,  losses  will  be  assumed  to  be 
approximately  linear.  However,  the  impact  on  various  results  of 
concavity  or  convexity  of  the  loss  function  will  be  discussed 
briefly  in  Chapter  VI . 

Scheduling  to  maximize  the  total  service  rendered  to  all 
users  is,  from  a  more  pessimistic  view,  equivalent  to  minimizing 
total  losses  to  all  users  due  to  delay.  This  goal  is  an  appro¬ 
priate  system  objective  both  selfishly  and  selflessly.  Service 
rendered  is  best  measured  by  the  amount  of  money  the  user  is 
willing  to  pay  for  that  level  of  service.  A  user's  loss  function 
indicates  how  much  he  is  willing  to  pay  for  completion  of  service 
at  various  points  in  time.  If  he  is  charged  the  maximum  he  is 
willing  to  pay,  system  income  is  maximized  simultaneously  with 
total  service  rendered.  If  service  is  not  accurately  reflected 
by  willingness  to  pay  (that  is,  if  money  is  not  equally  valuable 
to  all  users)  then  the  willingness-to-pay  functions  (or  loss 
functions)  must  be  weighted  by  the  relative  value  of  money  to  that 
user  in  order  to  maximize  total  service  rendered.  We  will  con¬ 
sider  further  only  the  case  in  which  money  is  an  accurate  common 
measure  of  value  to  all  users. 

Every  request  to  a  computer  system  is  accompanied  by 
certain  information,  either  explicit  or  implicit,  regarding  the 
character  of  that  request  .  Possible  items  of  information  given 
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explicitly  are  service  time  estimate,  maximum  service  time, 
relative  priority  class,  size  of  program,  project  number,  user  id, 
main  system  program  to  be  used  (which  compiler) ,  and  so  forth. 
Items  of  information  known  implicitly  are  source  of  request  (batch 
stream  or  on-line  terminal)  and  any  of  the  information  which  is 
sometimes  explicit.  In  the  case  of  a  sequence  of  requests  from 
an  on-line  terminal,  these  are  stated  upon  entrance  to  the  system 
and  known  implicitly  thereafter.  Such  information  may  be  used  to 
associate  each  arriving  request  with  a  particular  request  class. 

Perhaps  the  greatest  difficulty  in  scheduling  computer 
system  resources  is  the  unpredictability  of  how  long  a  request 
will  need  a  resource.  If  the  service  time  distribution  of  the 
requests  in  each  class  is  known,  the  information  can  be  used  to 
schedule  requests  with  a  decreased  expected  total  loss.  The 
choice  of  a  class  partitioning  function  is  an  important  degree  of 
freedom.  In  general,  we  wish  to  partition  requests  according  to 
characteristics  which  are  good  predictors  of  actual  service  time 
distribution.  Too  coarse  a  partition  fails  to  use  available 
information,  and  too  fine  a  partition  increases  the  volume  of 
information  which  must  be  preserved  and  may  detract  from  the 
statistical  validity  of  the  distributions  by  limiting  the  number 
of  observations  upon  which  they  are  based.  A  "best"  partitioning 
function  for  balancing  these  considerations  depends  on  the  partic¬ 
ular  request  mix  and  environment  of  an  installation. 

While  the  distributions  for  each  request  class  cannot  be 
actually  known,  they  may  be  estimated  empirically  by  observing 
recent  requests  in  each  class.  In  most  installations,  request  mix 
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varies  relatively  slowly.  Even  daily  or  weekly  cyclic  variations 
of  request  mix  may  be  taken  into  account  by  including  time  of 
submission  in  the  information  upon  which  the  class  partition  is 
based.  We  presume  the  service  time  characteristics  of  past  re¬ 
quests  of  a  particular  class  to  be  a  reasonably  accurate  predic¬ 
tion  of  the  service  time  characteristics  of  a  current  request  of 
the  same  class.  A  major  goal  of  this  research  is  the  demonstration 
of  the  potential  for  better  scheduling  by  using  knowledge  of  ser¬ 
vice  time  distributions. 

We  will  consider  various  request  arrival  patterns. 

Chapter  III  will  deal  with  a  finite  set  of  initially  available 
requests  with  no  arrivals  occurring.  In  Chapter  IV,  principal 
consideration  will  go  to  the  case  of  random  of  Poisson  distributed 
arrivals.  Many  actual  computer  facilities  have  arrival  patterns 
modelled  as  random.  However,  the  rate  of  arrivals  typically 
varies  with  time  of  day,  or,  in  the  case  of  on-line  service  to  a 
small  population  of  users,  with  the  number  of  requests  already  in 
system.  But  if  the  user  population  is  reasonably  large  and  the 
scheduling  horizon  is  short  relative  to  the  variation  in  arrival 
rate,  then  treating  arrivals  as  Poisson  for  the  purpose  of  sched¬ 
uling  is  justified.  With  the  current  trend  of  increasingly  large 
user  populations  being  served  by  a  system,  this  assumption's  valid¬ 
ity  grows.  Scheduling  under  more  general  arrival  patterns  will 
also  be  considered  briefly  in  Chapter  IV. 

Because  the  processor  is  assumed  to  be  the  critical  system 
resource,  the  overhead  of  preemption  can  be  measured  in  terms  of 
lost  processor  time.  The  presumption  that  this  cost  is  constant 
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for  all  preemptions  implies  no  dependency  on  rotational  device 
delay  or  program  size.  However,  preemption,  as  used  in  current 
multiprogramming  systems,  involves  only  the  interchange  of  process 
statewords  (storing  and  loading  central  registers) .  The  duration 
of  this  operation  is  nearly  constant  over  all  preemptions.  Pre¬ 
vious  results  for  models  in  which  preemption  is  disallowed  cor¬ 
respond  to  the  special  case  in  which  C-*00.  Chapters  III  and  IV 
will  assume  that  the  time  used  in  preemption  is  negligible. 

Chapter  V  investigates  the  case  of  non-zero,  but  tolerable, 
preemption  cost. 


CHAPTER  II 


A  SURVEY  OF  THE  RELEVANT  LITERATURE 

Previous  approaches  to  the  problem 

In  attacks  on  the  problem  of  computer  resource  scheduling, 
three  distinct  approaches  have  been  taken:  empirical  measurements, 
model  simulation,  and  model  analysis.  A  bibliography  and  dis¬ 
cussion  of  results  from  all  three  approaches  are  provided  by 
Schrage  [57] .  Scherr  found  that  empirical  measurements  confirmed 
the  predictions  of  both  analysis  and  simulation  in  his  study  of 
the  MIT  CTSS  System  [52] .  R.  L.  Smith  selected  a  scheduling  rule 
for  his  installation  after  an  investigation  which  involved  all 
three  approaches  at  different  stages  163] . 

Greenberger  enumerates  some  of  the  independently  desirable 
goals  of  scheduling  [25] : 

(1)  reduce  response  time  to  requests, 

(2)  reduce  number  of  requests  in  system, 

(3)  increase  thruput , 

(4)  acknowledge  request  priorities. 
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(5)  service  requests  in  a  "fair"  order, 
and  (6)  limit  the  maximum  waiting  time. 

Such  goals  can  be  classified  as  "system  oriented"  or  "user 
oriented."  Number  in  system,  and  thruput  are  system  oriented 
goals,  while  quick  response,  and  acknowledgement  of  priorities  are 
user  oriented  goals.  Some  of  these  goals  are  mutually  con¬ 
flicting  so  that  the  selection  of  an  overall  objective  function 
for  scheduling  involves  tradeoffs  among  them.  Many  objective 
function  candidates  have  been  proposed  [27,  48,  62],  but  the  one 
that  has  dominated  is  that  of  maximizing  the  total  utility  of 
service  provided.  One  reason  for  this  objective's  popularity  is 
the  inherent  ambiguity  in  its  definition.  Somehow  a  common 
measure  must  be  found  to  relate  the  utility  of  all  aspects  of 
service.  This  has  been  done  by  postulating  that,  associated  with 
each  request,  there  is  a  "delay  cost  curve"  or  the  total  disutil¬ 
ity,  penalty  cost,  loss  of  good  will,  opportunity  cost,  delay  of 
revenue,  customer  dissatisfaction,  and  so  forth  incurred  as  a 
function  of  the  request's  time  in  the  system  before  its  service 
is  completed  [25] .  The  goal  of  scheduling  is  then  to  maximize 
total  utility  by  minimizing  the  total  losses  to  all  users  as 
reflected  by  their  loss  functions.  Authors  who  have  used  loss 
functions  in  analytic  studies  of  scheduling  problems  include 
Greenberger  [25] ,  Schild  and  Fredman  153]  ,  McNaughton  [45]  ,  and 
Fife  [20] .  The  loss  functions  are  usually  assumed  to  be  linear. 

Previous  attempts  to  solve  models  closely  related  to 
MODEL  appear  in  the  literature  of  both  production  scheduling 
and  queueing  theory.  Techniques  from  both  areas  are  needed  to 
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attain  significant  results  in  the  area  of  resource  allocation 
scheduling  in  computer  systems. 

In  production  scheduling,  analytic  results  have  been  derived 
for  many  objective  criteria.  Among  these  are  scheduling  to 
minimize  average  time  in  system,  minimize  average  number  in 
system,  or  minimize  maximum  waiting  time.  The  number  of  scheduling 
rules  that  have  been  developed,  evaluated,  and  compared  with  re¬ 
spect  to  various  objective  criteria  is  large  [14,  27,  59,  63]. 

The  possibility  of  arrivals  receives  little  attention  in  the 
production  scheduling  approach. 

Queueing  theory  analyses  of  scheduling  problems  have 
provided  a  basis  for  a  vast  body  of  literature.  Saaty  presents 
an  extensive  bibliography  of  queueing  theory  results  up  to  1961 
[51,  pp.  375-413]  .  There  are  also  several  good  surveys  of  that 
portion  of  queueing  theory  literature  which  pertains  specifically 
to  resource  scheduling  in  computer  systems.  Some  of  these  are 
McKinney  143]  ,  Coffman  and  Kleinrock  [14]  ,  Coffman  [11] ,  Chang  [6] , 
and  Estrin  and  Kleinrock  [19] .  Queueing  theory  has  proven  its 
value  as  a  tool  in  the  analysis  and  evaluation  of  particular 
scheduling  strategies.  It  possesses  the  ability  to  deal  with  a 
system  with  arrivals  and  determine  steady  state  characteristics  of 
the  system.  In  parameterized  strategies,  its  results  can  indicate 
the  optimal  parameter  value  which  leads  to  a  locally  optimal 
scheduling  strategy.  However  proof  of  global  optimality  is  fre¬ 
quently  beyond  the  scope  of  queueing  theory  since  some  form  of 
combinatorial  argument  is  generally  required. 

Some  evaluation  measures  frequently  derived  by  queueing 
theory  are  expected  number  in  system  (at  steady  state  under 
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arrivals) ,  expected  waiting  time,  and  expected  waiting  time  con¬ 
ditioned  on  service  time.  Relation  to  particular  loss  functions 
is  typically  not  done.  This  allows  the  flexibility  of  applying 
various  optimization  criteria  to  the  basic  result. 

In  most  queueing  theory,  priority  is  considered  only  to  the 
extent  of  absolute  priority  among  request  classes.  Exceptions  are 
"discretionary  priorities"  as  discussed  by  Jaiswal  [30,  p.  161], 
Schrage  [58],  and  others.  Here,  the  preemption  decision  at  each 
arrival  is  based  on  the  attained  service  time  of  the  request 
currently  in  service. 

Queueing  theory  has  other  limitations.  Preemption  is  con¬ 
sidered  only  at  arrival  instants  and  at  the  termination  of  quanta 
whose  lengths  are  independent  of  the  characteristics  of  the  par¬ 
ticular  request.  Assumptions  of  exponentially  distributed  service 
times  occur  more  frequently  than  is  justified  in  computer  systems 
due  to  the  desirable  transform  properties  of  the  exponential  dis¬ 
tribution.  Queueing  models  become  quite  complex  when  such  con¬ 
siderations  as  preemption  costs  are  included.  Precise  analysis 
of  multiple  resource  scheduling  models  is  extremely  difficult, 
although  recently  developed  approximation  and  bounding  techniques 
may  be  helpful  [38] . 

Selection  rules  of  interest 

We  will  now  describe  some  particular  scheduling  strategies 
of  importance.  Many  scheduling  strategies  select  the  request  to 
be  serviced  next  on  the  basis  of  some  combination  of  the  fol¬ 
lowing  attributes  of  individual  requests: 

(1)  time  of  arrival  (a^ ) , 
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(2)  attained  service  time  (bj  (x) ) , 

(3)  actual  service  time  (tj)  , 

(4)  expected  service  time  (Ett^j), 

(5)  time  in  queue  without  service  (qj  (x) )  , 

(6)  rate  of  linear  loss  (kj) . 

We  define  a  selection  rule  as  a  scheduling  strategy  which  assigns 
the  processor  to  the  request  with  current  minimum  (by  convention) 
value  of  a  function  involving  only  attributes  of  the  individual 
request.  When  loss  functions  are  linear,  and  arrivals  are  Poisson, 
selection  rules  are  frequently  sufficient  to  attain  optimal  sched¬ 
uling.  However,  non-linear  loss  functions  may  remove  the  possi¬ 
bility  of  establishing  a  transitive  priority  relation  among  the 
requests.  The  desirability  of  assigning  a  particular  request  may 
depend  upon  the  loss  function  characteristics  of  the  other  requests 
in  the  system.  Because  the  practicality  of  a  scheduling  strategy 
depends  on  its  operational  efficiency  as  well  as  its  ability  to 
lead  to  good  schedules,  a  computational  overhead  of  scheduling 
which  is  strictly  linear  in  the  number  of  active  requests  en¬ 
hances  the  desirability  of  selection  rules. 

Specification  of  a  selection  rule  consists  of  the  selection 
function,  and  the  events  which  initiate  application  of  the  selec¬ 
tion  rule.  The  rule  may  be  applied  upon  completions  (comp), 
upon  completions  and  arrivals  Cc&a) ,  at  regular  predetermined 
intervals  (quant) ,  or  continually  Ccont) .  Table  1  describes 
those  selection  rules  of  relevance  to  this  research.  The  rules 
in  Table  1  have  been  examined  from  both  production  scheduling  and 
queueing  theoretic  points  of  view  although  the  loss  constant 
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TABLE  1 

SELECTION  RULES3 


Rule 

Selection 

Function 

Init. 

Times0 

Description 

RAND 

0 

comp 

Random  selection 

FIFO 

ai 

comp 

First  come,  first  served 

LIFO 

J 

“ai 

comp 

Last  come,  first  served 

PLIFO 

J 

-aj 

c&a 

Preemptive  last  come,  first 
served 

SPT 

t  j 

comp 

Shortest  processing  time 

SRPT 

t j-bj  (t) 

c&a 

Shortest  remaining  processing 
time 

SEPT 

E[tj] 

comp 

Shortest  expected  processing 
time 

SERPT 

E [t j] -bj (t) 

c&a 

Shortest  expected  remaining 
processing  time 

SPT/c 

t  j/kj 

comp 

SRPT/c 

[ti-bi  (t)]/k, 

E  [  t-j  ]  /k-;  J 

c&a 

(As  above,  only  with 

SEPT/c 

comp 

non-identical  loss  rates) 

SERPT/c 

[E  [t j]-^j (t) ]/kj 

c&a 

RR 

-q-i  (t) 

quant 

Round  robin 

FBnc 

Mxd j  (t)-qj  (t) 

quant 

Multi-level  queues  with  feed¬ 
back  to  lower  priority  queues 

PS 

-q-4  (t) 

cont 

Like  RR  with  quantum  length  -*  0 

PPSC 

Mxb j (t) -qj (t) 

cont 

Like  FBR  with  quantum  length  0 

at  every  queue  level 

Assume  all  ties  are  broken  randomly. 

b 

' Initiation  times  shown  represent  the  minimum  frequency 
with  which  the  rule  must  be  applied.  For  all  rules  except  RAND, 
LIFO,  RR,  FB^,  SPT,  SEPT,  SPT/c,  and  SEPT/c,  scheduling  would  be 
unaltered  by  the  continuous  application  of  the  rule. 

cThe  number  M  is  very  large  relative  to  b • (t)  or  q. (t)  for 
any  request.  This  has  the  effect  of  scheduling^by  FIFO  Within 
levels  of  attained  service . 
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rules,  SPT/c,  SEPT/c,  SRPT/c,  and  SERPT/c  are  primarily  from 
production  scheduling,  while  the  quantum  driven  and  processor 
sharing  rules,  RR,  FBR,  PS,  and  PPS ,  are  treated  primarily  in 
queueing  theory.  RAND,  FIFO,  and  SPT  are  frequently  used  as 
benchmarks  for  comparative  evaluation  of  more  complex  scheduling 
strategies . 

The  FBn  rule  has  many  variations.  When  a  request  which 
fails  to  complete  in  an  assigned  quantum  is  always  forced  into  a 
lower  priority  queue,  the  rule  is  denoted  by  FB^.  The  quantum 
length  assigned  may  depend  on  the  priority  level  of  the  queue 
being  served. 

Other  selection  rules  which  depend  on  time  in  system  are 
Jackson's  dynamic  priority  [29]  and  Kleinrock's  delay  dependent 
priority  [32] .  Coffman  and  Kleinrock  [14]  and  Mullery  and 
Driscoll  [46]  describe  scheduling  rules  designed  to  guarantee  a 
specific  response  time  to  short  requests  while  keeping  efficiency 
high  by  avoiding  unnecessary  preemptions  and  serving  batch  requests 
regularly.  Kleinrock  [36]  discusses  a  continuum  of  scheduling 
rules  based  on  a  two  parameter  scheduling  strategy.  Among  the 
strategies  encompassed  are  FIFO,  PLIFO,  and  RR. 

Uncertainty  of  service  times 

The  greatest  difficulty  in  scheduling  is  the  uncertainty  in 
knowledge  of  resource  needs.  Unfortunately,  when  dealing  with 
computer  systems,  the  knowledge  of  resource  needs  is  typically 
inversely  proportional  to  the  variety  of  services  provided  [27] . 

For  this  reason,  scheduling  in  specialized,  dedicated  systems  is 
typically  easier  than  in  general  purpose  computer  facilities. 
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In  MODEL  ,  the  relevant  uncertainty  is  that  in  the  service 
times  of  the  requests.  The  approach  frequently  taken  is  to 
derive  optimal  rules  assuming  perfect  service  time  knowledge, 
then  to  tune  or  adjust  the  rules  to  use  distributions  or  expec¬ 
tations.  Such  a  transition  is  that  from  SPT/c  to  SEPT/c. 

Optimal  perfect  information  rules  usually  perform  well  under 
partial  information  if  the  mean  service  time  is  assumed  to  be  the 
exact  service  time. 

Studies  by  Hellerman  127] ,  McKinney  [43] ,  Schrage  [59] ,  and 
Aczel  [1]  have  shown  that  rules  which  distinguish  among  requests 
on  the  basis  of  some  degree  of  knowledge  of  service  times  perform 
better  with  respect  to  expected  total  loss  than  rules  which  use 
no  knowledge  of  service  times.  Even  simple  two  priority,  non- 
preemptive  scheduling  with  service  times  estimated  as  "long"  or 
"short"  proves  much  superior  to  FIFO  when  service  times  are  dis¬ 
tributed  with  high  variance  [23,  60].  Schrage  suggests  that  the 
value  of  information  about  service  times  increases  with  system 
load  [57]  and  with  the  coefficient  of  variation  of  the  distri¬ 
bution  of  service  times  [59] . 

Scheduling  rules  which  use  service  time  information  may  be 
classified  according  to  whether  the  knowledge  is  available  when 
the  request  arrives  (SPT/c) ,  or  is  gained  only  as  the  request  is 
serviced  (RR) .  The  latter  group  is  called  "running  time  priority 
disciplines"  by  Coffman  and  Kleinrock  114] .  The  rules  using 
initially  available  service  time  information  are  always  better 
than  no  information  rules,  while  performance  of  the  running  time 
priority  disciplines  is  very  dependent  on  the  general  form  of  the 
service  time  distribution. 
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The  CTSS  scheduling  algorithm  uses  both  a  priori  information 
and  information  gathered  during  service  152] .  Because  service 
time  is  strongly  correlated  with  program  size  [57],  a  request's 
initial  priority  is  determined  by  its  size.  Small  programs  get 
high  priority  because  they  are  likely  to  be  short.  Thereafter, 
scheduling  is  done  according  to  an  FB^  rule,  with  the  quantum 
length  doubling  at  each  successively  lower  priority  queue. 

If  no  information  about  service  times  of  individual 
requests  is  available,  the  form  of  the  distribution  of  service 
times  is  important  when  selecting  a  scheduling  rule.  Many 
empirical  investigations  have  examined  service  time  distributions. 
In  computer  systems  which  provide  diverse  facilities  and  ser¬ 
vices,  the  distribution  of  service  times  has  been  found  to  have 
a  generalized  exponential  form:  exponential  [26]  ,  hyperexponential 
[21,  57] ,  log-normal  [57] ,  or  Waring  [7] .  R.  L.  Smith  presents  a 
thorough  statistical  profile  of  the  request  load  at  his  facility 
[63] ,  and  Freeman  and  Pearson  found  empirically  that  the  service 
time  distribution  at  their  installation  had  a  variance  to  mean 
squared  ratio  of  greater  than  six  [23] . 

Intuitively,  the  exponential  distribution  of  service  times 
is  an  appropriate  assumption  in  the  situation  in  which  no  knowl¬ 
edge  of  service  times  is  available  because  its  "memory less " 
property  [18,  p.  143]  reflects  a  "continuing  identical  uncer¬ 
tainty"  [14]  of  service  time  remaining.  This,  along  with  the 
empirical  studies  mentioned  earlier,  provide  justification  for 
the  assumption  of  exponentially  distributed  service  times  in 
analytic  and  simulation  studies  of  scheduling.  However,  the 
frequency  with  which  this  assumption  is  made  [15,  34,  41,  44,  49, 
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52,  61]  indicates  that  the  convenient  mathematical  properties  of 
the  exponential  distribution,  which  facilitate  the  acquisition  of 
results,  are  also  a  significant  motivation  for  assuming  exponen¬ 
tial  service  times. 

The  fact  that  empirically  determined  service  time  distri¬ 
butions  are  generally  exponential  in  form  indicates  many  short 
jobs  and  few  long  jobs.  FIFO  scheduling  is  "fair"  in  that  it 
equalizes  waiting  time  for  all  requests.  But  RR,  which,  with 
sufficiently  small  quanta,  makes  waiting  time  proportional  to 
request  service  time,  is  "fair"  in  another  sense.  Discrimination 
in  favor  of  short  jobs  has  been  generally  recognized  as  being 
inherently  "good."  Three  justifications  for  it  follow: 

1.  Short  requests  should  not  have  to  wait  for  service 
as  long  as  long  requests. 

2.  Since  short  requests  form  the  majority  of  all 
requests,  serving  them  well  keeps  the  majority  of 
users  happy. 

3.  Loss  rates  are  typically  higher  for  short  requests 
than  for  long  requests. 

The  rules  SPT/c,  SEPT/c,  SRPT/c,  and  SERPT/c  use  initially 
available  knowledge  of  service  times  to  discriminate  in  favor  of 
short  jobs.  SRPT/c  discriminates  more  than  SPT/c  through  its 
ability  to  preempt.  The  rules  RR,  FBn,  PS,  and  PPS  discriminate 
in  favor  of  short  jobs  on  the  basis  of  service  time  knowledge 
gained  by  experience  during  execution.  As  the  quantum  length 
decreases  in  RR  or  FBn,  approaching  PS  or  PPS,  the  degree  of 
discrimination  in  favor  of  short  requests  increases  [54] .  FBn 
and  PPS  discriminate  more  than  RR  and  PS,  respectively. 
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By  Kleinrock's  Conservation  Law  [33],  we  see  that  the  advan¬ 
tages  of  discrimination  in  favor  of  short  requests  come  only  at 
the  expense  of  longer  waiting  times  for  long  requests,  and  the 
associated  increase  in  variance  of  waiting  times.  The  Conservation 
Law  states  that  the  sum  over  all  job  classes  of  the  product  of 
the  load  factor  for  that  class  Carrival  rate  times  mean  service 
time)  and  the  expected  waiting  time  for  requests  of  that  class  is 
a  constant.  The  Law  has  at  least  two  important  implications  here. 
First,  with  a  single  job  class,  expected  waiting  time  is  a  con¬ 
stant,  and  linear  losses  imply  that  optimal  scheduling  requires 
selecting  the  request  with  highest  loss  rate.  With  identical 
linear  losses,  expected  total  loss  is  independent  of  service  dis¬ 
cipline,  and  scheduling  is  arbitrary.  Second,  for  multiple 
classes,  reducing  expected  waiting  time  for  one  request  class 
necessarily  involves  increasing  it  for  at  least  one  other. 

Along  with  the  availability  of  predictive  information  about 
service  times,  we  must  also  question  its  accuracy.  As  the 
accuracy  decreases,  the  performance  of  any  rule  approaches  that 
of  RAND,  or,  if  the  variance  of  service  estimates  is  small, 

FIFO  [52]  .  Some  authors  who  have  shown  special  concern  about 
uncertainty  in  service  time  knowledge  are  Banerjee  [4] ,  Fife  [20] , 
and  Rothkopf  [50] . 

Pertinent  previous  results 

The  scheduling  literature  contains  numerous  results  of  rel¬ 
evance  to  the  model  we  consider  here.  Simple  selection  rules  have 
been  proven  to  lead  to  optimal  scheduling  in  various  subcases  in 
which  losses  are  linear  and  preemption  is  free. 
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Result  1. — With  known  service  times,  preemption  disallowed, 
and  no  arrivals,  the  shortest  processing  time  discipline,  or 
SPT/c,  minimizes  total  losses. 

The  optimality  of  SPT/c  has  been  proven  by  W.  E.  Smith  [64] , 
McNaughton  [45],  and  others.  Two  important  lemmas  are  used  in 
McNaughton's  proof: 

1.  Preemption  is  not  necessary  when  service  times  are 
known . 

2.  The  processor  should  never  be  idle  when  work  is 
available . 

Fife  [20]  strengthens  the  latter  to  show  that  with  Poisson  arrivals 
and  no  preemption,  inserted  idle  time  leads  to  non-optimal  sched¬ 
uling  . 

Result  2. — With  distribution  of  service  times  known,  pre¬ 
emption  disallowed,  and  no  arrivals,  shortest  expected  processing 
time  discipline,  or  SEPT/c,  minimizes  expected  total  losses. 

The  optimality  of  SEPT/c  is  proven  by  Rothkopf  [50] ,  and 
others.  Aczel  proves  SEPT  optimal  among  static  priority  disci¬ 
plines  under  identical  linear  losses  and  Poisson  arrivals  [1] . 

Using  some  of  Cobham's  results  [8],  Fife  extends  SEPT/c' s  opti¬ 
mality  to  the  case  of  Poisson  arrivals  [20] . 

Result  3 . — With  remaining  service  times  known  precisely, 
preemption  allowed  and  identical  linear  loss  functions,  shortest 
remaining  processing  time,  or  SRPT,  minimizes  total  losses. 

The  optimality  of  SRPT  under  arbitrary  arrivals  is  proven 
by  Schrage  [55] .  In  order  to  remove  the  restriction  of  identical 
linear  losses,  we  must  restrict  the  pattern  of  arrivals.  With  no 
arrivals  (this  situation  occurs  after  the  last  arrival  of  a  busy 
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period) ,  SRPT/c  is  optimal  by  analogy  to  SPT/c.  Under  Poisson 
arrivals,  Fife  120]  judged  SRPT/c' s  optimality  to  be  intuitive 
but  difficult  to  prove.  Schrage  proposed  a  test  of  SRPT/c  under 
Poisson  arrivals.  The  test  results  show  SRPT/c  to  be  the  best  of 
a  class  of  strategies.  However,  it  has  not  been  shown  that  the 
optimal  strategy  must  be  in  the  class.  In  Chapter  IV,  the  test  is 
described,  and  we  will  attempt  to  resolve  the  question  of  SRPT/c' s 
optimality  under  Poisson  arrivals. 

Assertion  4. — With  distribution  of  service  times  known,  pre¬ 
emption  allowed  and  no  arrivals,  shortest  expected  remaining  pro¬ 
cessing  time  discipline  is  good,  but  not  necessarily  optimal,  with 
respect  to  minimizing  expected  total  losses. 

The  assertion  is  made  by  Schrage  [59] . 

Such  results  as  those  described  above  and  tabulated  in 
Table  2  are  typically  obtained  by  proofs  of  a  combinatorial  nature 
involving  an  interchange  argument.  The  interchange  of  processor 
assignment  in  adjacent  intervals  is  shown  to  reduce  the  expected 
total  loss  of  any  schedule  which  does  not  follow  the  appropriate 
selection  rule.  With  certain  assumptions  of  finiteness,  the  po¬ 
tential  for  improvement  in  every  schedule  which  does  not  follow 
the  selection  rule  indicates  that  only  schedules  which  do  follow 
the  selection  rule  can  be  optimal.  Note  that  all  the  optimal  se¬ 
lection  rules  described  above  involve  ranking  attributes  in  which 
the  numerator  depends  on  the  distribution  of  service  times  and 
the  denominator  is  the  loss  constant  of  the  request. 

Rothkopf  [50]  also  investigated  the  question  of  when  pre¬ 
emption  is  needed  to  attain  optimal  scheduling  in  the  case  of  no 


30 


TABLE  2 

OPTIMAL  SELECTION  RULES 


- - - - 

Service  times 
known 

Distribution  of  service 
times  known 

Preemption 

SPT/c  optimal 

SEPT/c  optimal 

disal lowed 

Preemption 

SRPT/c  optimal3 

SERPT/c  good  but  not 

allowed 

necessarily  optimal3 

aAfter  the  last  arrival  of  a  busy  period. 
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arrivals  and  non-deterministic  service  times.  He  cites 
McNaughton ' s  theorem  [45]  which  states  that,  on  a  single  pro¬ 
cessor,  exact  knowledge  of  service  times  makes  preemption  unnec¬ 
essary.  He  proves  that  with  linear  losses  and  exponentially 
distributed  service  times,  preemption  is  unnecessary  but  suggests 
that  other  distributions  may  motivate  preemption,  even  with  no 
arrivals.  He  suggests  the  need  for  more  general  preemption  pol¬ 
icies  than  have  been  studied.  Traditional  policies  have  consid¬ 
ered  preemption  only  at  arrivals  of  new  requests,  or  at  the  ter¬ 
mination  of  quanta  whose  lengths  are  independent  of  the  service 
time  distributions. 

In  trying  to  determine  what  service  time  distributions 
might  motivate  preemption  other  than  at  arrivals,  two  distri¬ 
bution  classification  schemes  from  the  field  of  reliability  theory 
have  been  used  [5] :  classification  by  monotone  mean  residual 
life,  and  classification  by  monotone  failure  rate,  which  is  some¬ 
times  called  hazard  rate  or  completion  rate.  The  mean  residual 
life  of  distribution  function  f ^  (x)  as  a  function  of  attained 
service  t  is  defined  as 


/  (x-t) f .  (x)  dx 

J  t  1  00 

MRLf  .  (t)  =  - - -  =  /  x  3  G.  ( x  ,  t )  . 

/  f^xjdx  0 

t 

This  is  precisely  expected  remaining  service  time  conditioned  on 
attained  service.  The  failure  rate  of  distribution  function  f^(x) 
as  a  function  of  attained  service  t  is  defined  as 
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This  is  the  likelihood  that  a  job  with  attained  service  t  will 
finish  imminently. 

A  distribution  is  said  to  be  of  decreasing  mean  residual 
life,  or  DMRL ,  if  its  mean  residual  life  is  monotone  non-increasing 
with  attained  service.  IMRL,  DFR,  and  IFR  have  the  corresponding 
meanings.  Rothkopf's  investigation  led  to  the  following  result 
[50]  . 

Result  5. — When  all  distribution  functions  are  DMRL,  pre¬ 
emption  at  times  other  than  arrival  instants  is  unnecessary  to 
minimize  expected  total  loss. 

Schrage  gives  several  optimality  results  which  are  condi¬ 
tioned  on  failure  rate  characteristics  of  the  distribution  of 
service  times  [59] : 

1.  PPS  is  optimal  for  identical  linear  losses  and  a 
DFR  service  time  distribution. 

2.  FIFO  is  optimal  for  identical  linear  or  convex 
downward  loss  functions  and  an  IFR  service  time  dis¬ 
tributions  . 

The  coefficient  of  variation  (ratio  of  standard  deviation 
to  mean)  is  yet  another  measure  which  has  been  used  to  classify 
distribution  functions  {59] .  For  exponential  based  distribution 
functions  (Erlang-k,  exponential  and  hyperexponential) ,  coefficient 
of  variation  makes  the  same  distinctions  as  failure  rate.  Coeffi¬ 
cients  of  variation  less  than,  equal  to,  and  greater  than  one 
imply  that  failure  rate  is  respectively  increasing,  constant,  and 
decreasing . 

The  work  of  Coffman  and  Muntz  [16]  indicates  a  stronger 
result  than  Rothkopf's.  Rothkopf  requires  monotone  non-increasing 
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expected  remaining  service  time  to  disallow  consideration  of  pre¬ 
emption.  Coffman  and  Muntz  present  evidence  that  the  restriction 
may  be  weakened  to  include  all  distribution  functions  whose  initial 
expected  remaining  processing  time  is  greater  than  or  equal  to  its 
expected  remaining  time  with  any  greater  attained  service  time. 

When  arrivals  are  allowed  the  result  extends  to  the  following. 

Result  6. — If  a  job's  expected  remaining  service  time  when 
assigned  the  processor  is  greater  than  its  expected  remaining  ser¬ 
vice  time  for  any  greater  amount  of  attained  service,  then  pre¬ 
emption  before  the  next  arrival  is  unnecessary  to  attain  opti¬ 
mality  . 

Konheim  [39,  40],  and  Chazan,  Konheim  and  Weiss  [7]  have 
treated  a  model  with  the  same  assumptions  made  in  MODEL Q  for  the 
case  of  free  preemption  and  no  arrivals.  They  remain  strictly  in 
realm  of  finite  mathematics  by  presuming  all  distribution  func¬ 
tions  to  be  discrete.  They  derive  certain  necessary  and  suffi¬ 
cient  conditions  for  a  given  schedule  to  be  optimal.  Identical 
linear  losses  and  only  two  jobs  are  considered  in  Chazan  et  al. 
Konheim  extends  to  an  arbitrary  number  of  jobs  and  non-identical 
linear  losses.  Konheim  represents  schedules  in  a  form  isomorphic 
to  a  binary  tree  with  each  node  representing  assignment  of  the 
processor  to  a  certain  request  for  a  certain  interval.  They  do 
not,  however,  specify  any  intuitive  method  or  selection  disci¬ 
pline  for  creating  an  optimal  schedule.  Konheim  proves  that 
minimizing  total  loss  under  identical  linear  losses  is  equivalent 
to  minimizing  the  time  to  next  completion  [39] ,  and  that,  with  no 
arrivals,  an  optimal  schedule  for  completing  m  requests  can  be 
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synthesized  from  the  ^ j  pairwise  optimal  two  request  schedules 
[40].  Chazan,  Konheim,  and  Weiss  present  some  examples,  some  of 
which  relate  to  a  distribution  function  classification  equivalent 
to  that  by  monotone  failure  rate.  The  most  significant  theorem 
proves  RR  optimal  for  scheduling  two  jobs  initially  available  when 
they  are  from  a  service  time  distribution  of  decreasing  failure 
rate . 

Much  empirical  investigation  122,  28,  57]  and  some  analytic 
research  [2,  41,  44]  has  been  devoted  to  determining  optimal 
quantum  lengths  for  various  quantum-driven  scheduling  techniques 
when  preemption  overhead  is  not  negligible.  Under  round- robin 
scheduling,  consensus  is  that  there  is  a  threshold  from  which 
decreasing  quantum  length  causes  disastrous  performance  degradation, 
although  increasing  the  quantum  length  above  the  threshold  only 
slowly  degrades  response  time  [44] . 

Schrage  has  given  some  consideration  to  the  non-linear  loss 
situations  when  there  is  no  advance  information  about  service 
times  [59].  His  assertions  include  the  following. 

1.  Identical  convex  losses  and  no  preemption  implies 
FIFO  is  optimal . 

2.  Identical  convex  losses,  preemption,  and  IFR 
distribution  implies  FIFO  is  optimal. 

3.  Identical  convex  losses,  preemption,  and  DFR  distri¬ 
bution  implies  RR  is  "good"  but  not  optimal. 

Identical  concave  losses,  preemption,  and  constant 
FR  implies  PLIFO  is  optimal. 


4. 
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Topics  of  later  chapters 

In  the  remaining  chapters,  new  results  are  derived  and  pre¬ 
sented  in  relation  to  the  results  described  above. 

Chapter  III  deals  with  the  case  of  linear  losses,  free  pre¬ 
emption,  no  arrivals,  and  service  time  distributions  known.  An 
obvious  candidate  for  the  optimal  scheduling  rule  which  is  based 
on  expected  remaining  processing  time  is  shown  to  be  inadequate  to 
attain  minimum  expected  total  loss,  and  a  more  general  preemption 
strategy  is  developed  for  a  certain  class  of  distribution  func¬ 
tions.  A  new  scheduling  rule  called  "Smallest  Rank",  or  SR  is 
defined  and  proven  optimal  with  respect  to  minimizing  expected 
total  loss,  and  its  compatibility  with  previously  solved  subcases 
is  demonstrated. 

In  Chapter  IV,  we  consider  the  effect  on  optimal  scheduling 
strategies  of  allowing  arrivals.  We  verify  SRPT/c's  optimality 
among  selection  disciplines  under  Poisson  arrivals,  known  service 
times  and  non-identical  linear  losses,  and  show  that  a  similar 
argument  indicates  SR's  optimality  among  selection  disciplines 
under  Poisson  arrivals.  Finally,  optimal  scheduling  under  more 
general  arrival  patterns  is  considered. 

Chapter  V  considers  the  impact  of  significant  preemption 
costs.  Two  particular  simple  systems  are  analyzed,  then  some 
necessary  characteristics  of  rules  which  optimize  under  significant 
preemption  costs  are  discussed. 

In  Chapter  VI  several  other  possible  generalizations  are 
considered  including  non-linear  loss  functions  and  the  relation¬ 
ship  of  some  queueing  theory  results  to  generalized  SR  scheduling. 

A  summary  of  results  and  conclusions  completes  Chapter  VI. 


CHAPTER  III 


AN  OPTIMAL  SELECTION  RULE  UNDER 
FREE  PREEMPTION  AND  NO  ARRIVALS 

The  Failure  of  SERPT/c 

In  this  chapter  we  assume  linear  losses,  free  preemption, 
and  no  arrivals.  As  was  pointed  out  in  Chapter  II,  when  pre¬ 
emption  is  free  and  service  time  distributions  are  known,  no 
scheduling  rule  has  been  proven  optimal.  After  examining  Fig¬ 
ure  2,  intuition  and  considerations  of  symmetry  lead  to  SERPT/c 
(smallest  ratio  of  expected  remaining  processing  time  to  loss 
rate)  as  an  initial  candidate  for  the  optimal  scheduling  rule. 
However,  we  may  generate  a  counterexample  to  SERPT/c' s  optimality 
as  follows.'*' 

Let  the  request  set  consist  of  one  request  from  each  of  two 
request  classes.  Class  1  requests  are  either  short  (service  time 
e)  with  probability  p,  or  they  are  long  (service  time  3)  with 
probability  1-p.  All  class  2  requests  have  service  time  a,  where 
e  <  a  <  3/  and  e/p  <  a  <  3~e .  Also  we  require  that  3  be  suffi- 

■*'The  possibility  of  generating  a  counterexample  was  sug¬ 
gested  by  L.  E.  Schrage. 
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ciently  large  that  a  <  pe  +  Cl-p) 8-  Assume  that  requests  from 
both  classes  have  unit  loss  rates.  Then  the  initial  expected 
remaining  service  times  for  the  two  classes  are 

E  [  t]_J  =  pe  +  Cl-p)  8  , 

E[t2J  =  a. 

We  selected  3  so  that  a  <  pe  +  Cl-p) 8 /  so  that  SERPT/c  strategy 
indicates  that  the  class  2  request  should  be  assigned  before  the 
class  1  request. 

But  consider  an  alternative  strategy:  Run  the  class  1 
request  for  e  time  units.  If  it  completes  (as  it  will  with  proba¬ 
bility  p)  simply  run  the  class  2  request.  If  it  fails  to  complete, 
preempt  it,  and  run  the  class  2  request  before  finishing  it. 

Under  the  first  strategy,  one  request  finishes  after  a,  and 
the  other  after  a  +  e  with  probability  p,  and  after  a  +  8  with 
probability  1-p .  Thus  the  expected  total  cost  is  2a  +  pe+(l-p)8* 
Expected  total  cost  is  lower  under  the  second  strategy  if  and 
only  if 

2a  +  pe  +  (1-p) 8  >  p{2e+a]  +  Cl-p) 1 2  (e+a)  +  C8-e) ] r 

a  >  e  +  Cl-p)  a, 
or 

a  >  e/p. 

We  picked  a,  e,  and  p  so  that  this  inequality  holds.  Thus  a 
strategy  other  than  SERPT/c  yields  lower  expected  total  cost  in 
this  situation.  One  set  of  parameter  values  which  leads  to  a 
counterexample  to  the  optimality  of  SERPT/c  is 


P 


.  5  ,  a  =  5  , 


e  =  1,  and  8  =  10 . 
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A  Case  study 

In  generating  a  counterexample  to  the  optimality  of  SERPT/c, 
the  scheduling  strategy  which  proved  superior  employed  preemption 
of  a  variety  which  has  gone  untreated  in  previous  literature  in 
the  field;  namely,  preemption  due  to  dynamic  prediction  of  the 
request's  characteristics.  Preemption  at  the  arrival  of  a  new 
request  to  the  system  has  been  treated  extensively,  as  has  regu¬ 
lar  (quantum  controlled)  preemption  such  as  is  used  in  time¬ 
slicing  systems.  Here,  we  wish  to  investigate  how  estimates  of 
service  time  distributions  gathered  empirically  may  be  fully 
exploited  in  scheduling  for  minimum  expected  total  loss. 

Consider  the  problem  of  scheduling  one  request  from  each  of 
two  classes  with  respective  cumulative  service  time  distributions 
F-^Cx)  and  F2  (x)  .  Assume  each  request  has  unit  loss  rate  and  the 
expected  service  time  of  class  2  requests  is  less  than  that  of 
class  1  requests: 

00  oo 

Ettj]  =  /  x  dF-^Cx)  >  Elt2]  =  /  x  dF2  (x)  . 

0  0 

Then,  if  preemption  is  disallowed,  the  optimality  of  the  SEPT/c 
selection  rule  dictates  that  the  class  2  request,  j2,  should  be 
run  first.  But  it  must  be  determined  whether  there  are  conditions 
under  which  the  class  1  request,  j-^,  should  be  selected  initially. 
Let  us  compare  the  following  particular  strategies: 

1.  Run  j2  to  completion,  then  run  j to  completion. 
Assign  j ^  for  a  quantum  of  length  e.  If  it  com¬ 
pletes,  run  to  completion,  otherwise  preempt  j-^ 
and  run  j 2  to  completion  before  finishing 


2. 
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Because  the  expected  length  of  exceeds  that  of  j2  >  strategy  1 
dominates  the  strategy:  Run  to  completion,  then  run  j 2  to 
completion . 

The  expected  total  loss  under  strategy  1  is 

00  00 

/  /  (2z+y) dF,  Cy) dF9 tz) , 

00 

and  under  strategy  2  is 

00  0  00  00 

/  /  (2y+z)dF1  (y)dF9(z)  +  /  /  [2  (z+e)  +  (y-e)  J  dF..  Cy)  dF0  Cz)  . 

0  0  1  ^  0  £  1  Z 

Strategy  2  is  better  than  strategy  1  with  respect  to  minimizing 
expected  total  loss  if  and  only  if 


or 


/  /  (2z+y) dFj Cy) dF2 Cz) 

CO  0  00  00 

>  /  /  (2y+z) dF1 Cy) dF2 Cz)  +  /  /  C2z+y+e) dF1 Cy) dF2 Cz) , 


0  0 


0  £ 


0  0 


-  -  00  £  -  — 

/  /  z  dF  Cy)dF  Cz)  >  /  /  y  dFnCy)dF2Cz)  +  /  /  Cz+e )  dF-.  Cy)  dF  C z) 

ft  ft  - L  z  00X^  -  -  Z 


0  £ 


I  z  dF2Cz)  >  /  y  dF 


rr-icy)l 

/  z  dF  Cz) 

L  e  J 

L  0  J 

+  e/  dF-L  Cy)  , 

£ 


[  fS 


Cy) 


-] 


J  z  dF2  Cz)  I  >  /  y  dF2Cy)  +  ef  dFx  Cy) 


/  z  dF  2 ( z )  > 


/  y  dFx(y)  +  e/  dFx  (y) 


CD 


/  dF1Cy) 


Thus  strategy  2  is  better  than  strategy  1  whenever  we  can  find  a 
value  for  e  such  that  inequality  Cl)  holds.  An  examination  of 
inequality  Cl)  for  intuitive  meaning  reveals  that  the  numerator 
of  the  righthand  side  is  precisely  the  expected  amount  of  time 
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would  run  if  it  were  assigned -for  a  quantum  of  length  e.  Since 

e  e  J  y  dFi(y) 

/  y  dFnty)  =  /  dF-,Cy)-^ - , 

0  Q  , 

I  <3F,(y) 

0 

the  first  term  of  the  righthand  side  is  the  product  of  the  proba¬ 
bility  that  will  complete  by  e  and  the  expected  length  of  a 
request  with  service  time  less  than  or  equal  to  e.  The  second 
term  is  the  product  of  e  and  the  probability  that  j  will  not 
complete  by  £ .  The  denominator  of  the  righthand  side  is  simply 
the  probability  that  j  will  complete  by  e.  Thus,  the  righthand 
side  is  the  ratio  of  the  expected  time  invested  to  the  probability 
of  completion  in  scheduling  j ^  for  e  time  units.  Note  that  simi¬ 
larly  the  lefthand  side  is  the  ratio  of  expected  time  invested  to 

the  probability  of  completion  in  scheduling  j  to  run  to  com- 

2 

pletion.  Also  note  that  we  are  free  to  choose  e  so  that  the 
ratio  of  expected  time  invested  to  probability  of  completion  is 
minimized.  Selecting  a  value  of  e  which  will  minimize  the  quan¬ 
tity  on  the  righthand  side  requires  knowledge  of  the  service  time 
distribution,  F^(y).  Thus  a  greater  degree  of  knowledge  is 
required  with  preemption  allowed  than  with  preemption  disallowed, 
where  a  rule  based  only  on  expected  service  time  minimizes 
expected  total  loss. 

Formation  of  a  new  rule 

Based  on  the  above  calculation  of  how  to  minimize  the 
total  loss  of  two  requests  with  at  most  one  preemption,  we  will 
now  construct  a  new  candidate  for  the  optimal  scheduling  rule 
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with  preemption  allowed  and  distribution  of  service  times  known 
for  each  request  class.  We  now  state  several  new  definitions. 

(a)  Expected  investment  and  expected  payoff.  Assume  that 
request  j  of  class  i  has  received  t  units  of  service  without 
completing.  Consider  assigning  to  request  j  as  much  as  it  needs 
of  the  next  £  units  of  processor  time.  Let 

£  oo 

I(j,t,e)  =  /  x  8xGiCx,t)  +  e[  8xGitx,t), 

0  £ 

=  the  expected  investment  of  processor 
time  in  allowing  j  up  to  e  more  units 
after  it  has  attained  t  units  of  service 
without  completing, 

e 

and  P(j,t,e)  =  /  a  Gi(x,t), 

o 

=  the  expected  payoff  in  allowing  j  up  to 
£  more  units,  which  is  the  probability 
that  j  will  complete  in  its  next  £ 
units  of  service. 

(b)  Interval  rank.  Let 


P ( j ,t,£) 


I  Cj  ,t,s) 

kjP (j ,t , e) 

lim  J ( j , t , 6 ) 
6+0k_.P(j  ,  t ,  6 ) 


if  £  >  0 
if  £  =  0, 


the  interval  rank  of  assigning  j  a 


quantum  of  length  £. 


Note  that 
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This  limit  is  finite  if  and  only  if  the  mean  of  the  distribution 
is  finite.  The  limits  used  in  the  definition  of  interval  rank 
are  discussed  in  Appendix  I. 

(c)  Rank.  Let 


i?Cj#t)  = 


=  the  rank  of  request  j  with  attained 
service  t. 

(d)  Rank  quantum.  Let 

Q(j,t)  =  max  [e  |  R  {,  j,t,e)  =  5(j,t)], 

=  the  rank  quantum ,  or  the  interval  for 
which  j  should  be  assigned  in  order  to 
minimize  the  ratio  of  investment  to 
payoff . 

Note  the  close  relationship  of  the  definition  of  rank  to 
the  procedure  used  earlier  of  picking  a  value  for  £  which  mini¬ 
mizes  the  ratio  of  time  invested  to  the  likelihood  of  completion. 
Intuitively,  the  k^  appears  in  the  denominator  of  the  definition 
of  interval  rank  because  the  "payoff"  of  a  completion  is  propor¬ 
tional  to  the  rate  at  which  the  utility  of  completing  the  request 
is  declining. 

(e)  Smallest  rank  scheduling.  Rank,  as  defined  above,  may 
be  used  as  a  ranking  attribute.  We  call  the  associated  selection 
rule  "smallest  rank"  or  "SR"  scheduling: 

Under  SR  scheduling,  at  every  instant,  the 
processor  is  assigned  to  the  request  of  current 


smallest  rank. 
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Relation  of  SR  to  other  rules 

Before  attempting  to  prove  the  optimality  of  SR  scheduling, 
we  will  relate  it  to  the  optimal  decision  rules  for  the  special 
cases  of  MODEL 0  discussed  in  Chapter  I.  In  order  to  relate  SR 
scheduling  to  the  case  in  which  preemption  is  disallowed,  we 
extend  the  definition  of  rank  as  follows: 

i?Cj/t,e)  if  preemption  is  allowed 


P(j,t)  = 


p^oo  PCj,t,e)  if  preemption  is  not  allowed. 

i  e 

This  reflects  the  fact  that  when  preemption  is  not  allowed,  we 
may  only  run  requests  to  completion.  That  is,  they  must  be 
assigned  for  a  quantum  of  infinite  length. 

In  the  special  case  of  service  times  known  and  no  preemption 
allowed,  SPT/c  is  the  optimal  selection  rule.  Consider  rank: 


i?(j,t)  =  P(j,t,°°) 


I (i  /t,°°)  =  ^2 

k  jP  ( j  ,  t ,°°)  k_. 


Therefore  SR  specializes  to  SPT/c. 

In  the  special  case  of  service  time  distributions  known  and 
no  preemption  allowed,  SEPT/c  is  optimal.  Consider  rank: 


P  Cj  ,t)  =  R  Cj  ,t,°°) 


/  x8xGi(x,0) 
I  ( j  / 1,°°)  _  _o _ 

k  jP  ( j  ,  t ,°°)  (kj)  (1) 


E 


Therefore  SR  specializes  to  SEPT/c. 

In  the  special  case  of  service  times  known  and  preemption 
allowed,  SRPT/c  is  optimal  after  the  last  arrival  of  any  busy 
period.  Consider  rank: 


P(j,t)  = 


mm 

£>0 


R  ( j ,t,e)  = 


[■ 


I ( j ,t,e) 


kjP( j , t,e) 


]■  -4^] 
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Therefore  SR  specializes  to  SRPT/c.  With  identical  linear  losses, 
SR  and  SRPT  lead  to  identical  processor  assignment  functions. 


The  Dynamics  of  rank 

The  rank  of  a  request  varies  with  its  attained  service  time 
as  do  other  distribution  based  measures  such  as  expected  remaining 
service  time  (mean  residual  life),  failure  rate,  and  coefficient 
of  variation  of  the  conditional  distribution.  We  reiterate  the 
definitions  of  these  measures. 

(a)  Rank. 

e  00 


R  (j  ,  t) 


/  x  8xGi(x,t)  +  e/  9xGi(x,t) 

1  min  _o _ £ _ 

k.  e>0  e 

3  /  9xGi(x,t) 


where  request  j  is  in  request  class  i. 

(b)  Expected  time  remaining  or  mean  residual  life. 


00 

ETR(t)  =  MRLtt)  =  /  x  9xG±(x,t). 

o 


(c)  Failure  rate. 


FR  ( t)  = 


fj_  (t) 
1“F± (t) 


=  gi  (x  ,t)  . 


(d)  Coefficient  of  variation. 


CV  ( t)  = 


j7  x2  dxG±ix,t)  -  [ETR  (t)  ]  2J 


1/2 


ETR  (t) 

Two  types  of  schemes  for  classifying  distribution  functions 


^The  coefficient  of  variation  is  sometimes  defined  as  the 
square  of  this  quantity,  which  is  variance  over  the  square  of  the 
mean . 
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have  been  built  on  these  measures.  One  deals  with  monotonicity  of 
the  measures  as  attained  service  increases,  while  the  other  deals 
with  the  point  at  which  the  measure  attains  its  maximum  over  all 
values  of  attained  service.  Extending  Rothkopf's  definitions 
[50],  distributions  whose  mean  residual  life,  failure  rate,  and 
rank  are  monotone  non-increasing  with  attained  service  will  be 
denoted  by  DMRL,  DFR,  and  DR  respectively.  Similarly,  IMRL ,  IFR, 
and  IR  denote  distributions  with  the  respective  measures  mono¬ 
tone  non-decreasing.  When 

ETR(O)  >  ETR(t)  t  >  0, 

the  distribution  will  be  called  SUPETR,  and  SUPR  implies 

R  C  j  ,0)  >  R  Cj  , x)  t  >  0  . 

Clearly,  DMRL  implies  SUPETR. 

Because  conditional  distributions  are  equivalent  to  uncon¬ 
ditional  distributions  of  a  different  random  variable,  we  may  say 
that  a  conditional  distribution  is  SUPETR  if  its  expected  time 
remaining  with  that  much  attained  service  is  not  exceeded  by  its 
expected  time  remaining  with  any  greater  amount  of  attained 
service.  A  similar  statement  holds  for  SUPR. 

Consider  first  a  generalized  exponential  family  of  distri¬ 
butions  including 

Cl)  exponential:  f  (t)  =  Xe”^t,  t  >  0, 

n  -x  •  t 

(2)  hyperexponential:  f Ct)  =  £  p< X -e  1  ,  t  >  0, 

i=l 


n 

where  j  p.  =  1  and  p.  >0,  i=l,2,...,n, 
i=l  1  1 
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(3)  Erlang-k:  f  (t)  =  — - - - — ,  t  >  0, 

Ck-1)  1 

with  integer  k  >  1. 

The  exponential  distribution  is  a  special  case  of  both  the 
Erlang-k  (for  k=l)  and  the  hyperexponential  (for  n=l  or  A.j_=Aj 
i,j).  Note  that  all  the  distribution  density  functions  are 
continuous.  The  exponential  has  the  distinction  of  being  the 
only  continuous  distribution  for  which  MRL (x) ,  FR(x) ,  #(j,x) ,  and 
CV(x)  are  all  independent  of  the  attained  service  time,  x.  This 
is  due  to  its  "memoryless  property"  which  indicates  that  its 
conditional  distribution  is  independent  of  the  attained  service 
upon  which  it  is  conditioned  118] .  For  an  exponential  distri¬ 
bution  with  parameter  A, 

ETR(t)  =  MRL  (t)  =  1/A, 
i?(j,t)  =  1/kj  A  , 

FRtt)  =  A, 
and  CVtt)  =  1. 

Note  that  these  quantities  are  all  independent  of  t.  The  expo¬ 
nential  distribution  is  both  SUPETR  and  SUPR. 

For  k  >  1,  Erlang-k  distributions  may  be  shown  to  have 
CV(t)  <  1  and  to  be  DMRL ,  IFR,  DR,  SUPETR,  and  SUPR.  For  the 
case  k=2, 

ETR(t)  =  MRL ( t)  =  (t+l/A)e“2At, 
i?(j,t)  =  (t+l/A)e"2Xt/k.  , 

FR  (t )  =  A2t/ (1+At)  , 
and  CV(t)  =  1//2. 

and  A^  /  A .  for  some  i  and  j ,  the  hyperexponential 


For  n  >  1 
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distribution  is  IMRL,  DFR,  IR,  and  neither  SUPETR  nor  SUPR.  If 
n=2,  p^=p2=l/2,  and  A^=n  >  then 

ie-^t  +  ie”tt 

ETR(t)  =  MRL  (t)  =  I! - H - , 

e-nt  +  e-yt 


#(j,t)  = 


-T]  t  -yt 
e  +  e 

k..  (ne"nt  +  ye_yt) 


FR  ( t)  = 


rie  Tlt  +  ye 

e-nt  +  e-yt  ' 


and  CV(t)  =  /2- 


l_  -2nt 

n2 


-&*?> 


-  (n+y ) t  +  l  e-2yt 


1  -nt  ,  l  -yt 

n«  +  ye 


Within  these  generalized  exponential  distribution  functions, 
the  set  diagram  in  Figure  2  shows  the  distinctions  we  have  dis¬ 
cussed.  The  lone  member  of  the  intersection  is  the  exponential 
distribution  which  has  CV (t)  =  1  and  constant  FR,  MRL,  and  R. 

More  generally,  for  all  continuous  distribution  functions, 
the  set  diagram  is  shown  in  Figure  3.  Outside  the  family  of  gen¬ 
eralized  exponential  distributions,  the  significance  of  coeffi¬ 
cient  of  variation  is  unclear. 

Discrete  distribution  functions  of  more  than  one  non-zero 
mass  point  can  have  monotone  rank,  monotone  mean  residual  life, 
and  monotone  failure  rate  only  if  we  carefully  observe  values 
only  at  the  discrete  mass  points.  We  define  here  special  ver¬ 
sions  of  the  rank,  mean  residual  life,  and  failure  rate  functions 
to  provide  useful  distinctions  among  discrete  distribution  func¬ 
tions  which  have  mass  points  at  integral  multiples  of  a  basic 
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Figure  2. — Dynamic  characteristics  of  generalized  expo¬ 
nential  distributions. 


Figure  3. — Dynamic  characteristics  of  continuous  distri¬ 
butions  . 
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time  unit,  A.  Let  p^(k)  be  the  probability  that  a  class  i 
request  has  service  time  kA  where 


00 


k=l 


I  P±  Ik)  =  1. 


We  define 


00 


l  j Ap± (k+ j ) 


ETRA(k)  =  MRLA(k) 


00 


j=l 


l  Pi (k+ j ) 


FRA(k) 


P±  (k+1) 


00 


l  Pi (k+ j ) 
j=l 


and  R ( j ,kA) 


f  •  •  • 


Not  too  surprisingly,  examination  of  Figure  4  shows  that 
the  distinctions  among  discrete  distributions  correspond  directly 
to  those  among  continuous  distributions.  Note  that  the  exponen¬ 
tial  distribution  has  been  replaced  by  its  discrete  analog,  the 
geometric  distribution,  as  the  single  distribution  which  is  IFR^ , 
DFRa,  IRa,  DRa,  IMRLa,  and  DMRL A . 

For  their  purposes  in  reliability  theory,  Barlow  and 
Proschan  use  classification  by  monotone  failure  rate  as  a  basis 
of  their  work  [5] .  In  scheduling,  however,  results  which  have 
been  stated  in  terms  of  failure  rate  appear  equally  valid  for  the 
broader  classes  of  monotone  mean  residual  life  distributions. 

Some  distribution  functions  are  neither  discrete  nor  con¬ 
tinuous.  For  example,  consider 


F  (x) 


{1  -  e  0<x<M 

1  x>M. 


x>M. 
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Figure  4. — Dynamic  characteristics  of  discrete  distri¬ 
butions  . 
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Such  a  distribution  occurs  if  a  time  limit  is  enforced  on  requests 
which  have  exponentially  distributed  service  times.  For  distri¬ 
butions  which  are  neither  discrete  nor  continuous,  SUPETR  implies 
and  is  implied  by  SUPR. 

By  combining  some  of  these  relationships  with  some  of  the 
results  described  in  Chapter  II,  we  may  judge  SR's  compatibility 
with  scheduling  rules  previously  known  optimal  in  certain  situ¬ 
ations.  In  each  of  the  following  cases,  assume  a  system  with  a 
single  request  class. 

Case  1. — DMRL  implies  no  preemption  needed  [50]  and  DMRL 
implies  DR.  DR  is  compatible  with  no  preemption  since  with  a  DR 
distribution,  SR  runs  every  request  to  completion. 

Case  2 . — SUPETR  implies  no  preemption  needed  [16]  and 
SUPETR  implies  SUPR.  SUPR  is  compatible  with  no  preemption  since 
with  a  SUPR  distribution,  the  rank  of  an  unstarted  request  is 
always  at  least  as  high  as  that  of  a  started  request,  so  SR  runs 
every  request  to  completion. 

Case  3 . — DFR  implies  PPS  is  optimal  [59]  and  DFR  implies  IR. 
PPS  scheduling  is  compatible  with  IR  since  with  rank  monotone 
non-decreasing,  those  requests  with  least  attained  service  are 
among  those  of  lowest  rank. 

Case  4. — IFR  implies  FIFO  is  optimal  [59]  and  IFR  implies 
DR.  FIFO  scheduling  is  compatible  with  DR  since,  as  was  seen  in 
case  2 ,  a  DR  distribution  leads  SR  to  specialize  to  non-preemptive , 
or  FIFO,  scheduling. 

Case  5. — Exponential  distribution  implies  any  schedule  is 
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optimal  [33]  and  exponential  distribution  implies  constant  rank. 
Arbitrary  scheduling  is  compatible  with  constant  rank  since  with 
a  constant  rank  distribution,  all  requests  always  have  the  same 
rank  independent  of  attained  service,  and  any  schedule  is  an  SR 
schedule . 

Thus ,  all  the  above  situations  are  special  cases  of  SR 
scheduling . 

Less  orderly  distribution  functions  than  those  discussed 
above  have  intervals  and  points  of  attained  service  at  which 
rank,  along  with  mean  residual  life,  increases  among  intervals  of 
decreasing  rank.  For  example,  consider  again  the  two-spike 
distribution  used  in  the  counterexample  to  SERPT/c's  optimality. 
Figure  5 (A)  shows  the  density  function  of  the  distribution  and 
Figure  5(B)  shows  a  graph  of  rank  versus  attained  service  time. 

By  the  definition  of  rank. 


F(j,t)  = 


{min  I 
6  -  t 


or  i?(j,t)  = 


min  I(e-t)/p,  pe+Cl-p)3-t]  0<t<e 

e<t<$ , 

ps+ (1-p) j3-t  0  <  t  <  (1+p) e-pB 

(e-t)/p  (1+p) e-pB  <  t  <  e 

B  -  t  e<t<$. 

In  Figure  5(B) ,  note  that  rank  is  decreasing  except  at  the  point 
at  which  e  units  of  service  is  attained.  Under  SR  scheduling, 
preemption  can  occur  only  at  points  of  increasing  rank.  For  a 
given  distribution,  we  can  identify  those  points  or  intervals  of 
increasing  rank.  Only  at  those  points  or  during  those  intervals 
is  preemption  potentially  desirable. 
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f  (t) 


(A)  The  density  distribution  function. 


(B)  Rank  versus  attained  service. 

Figure  5. — Rank  versus  attained  service  for  a  two-spike 
distribution . 
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The  Optimality  of  SR  scheduling 

We  now  prove  the  optimality  of  SR  scheduling.  For  the 
present,  two  assumptions  of  finiteness  will  be  made :  ^ 

1.  Assume  that  the  processor  is  assigned  only  for 
intervals  which  are  integer  multiples  of  some 
basic  unit. 

2.  Assume  that  all  service  time  distributions  are 
finite.  That  is,  for  each  request  class,  i, 

M .  <  °°  such  that  F.  (M  •  )  =  1. 

Because  actual  processors  of  interest  do  possess  basic  cycle 
times,  and  because  practicality  makes  it  prudent  to  set  some 
limit  on  service  times  for  any  request  class,  these  assumptions 
in  no  way  hinder  the  applicability  of  the  theorem.  However, 
modifications  and  extensions  necessary  in  the  proof  in  order  to 
eliminate  the  need  for  the  finiteness  assumptions  will  be  dis¬ 
cussed  after  the  proof  itself. 

The  proof  is  composed  of  two  preliminary  lemmata,  a  main 
lemma,  and  a  theorem.  The  first  two  lemmata  relate  various 
interval  ranks  of  a  single  request  as  it  receives  service.  The 
main  lemma  shows  that  a  schedule  can  be  optimal  only  if  it  assigns 
the  request  of  current  smallest  rank  first  for  at  least  its 
current  rank  quantum  by  showing  a  way  of  improving  any  schedule 
which  does  not.  A  theorem  then  states  the  main  result  of  this 
chapter:  SR  scheduling  minimizes  expected  total  losses  under 

free  preemption  and  no  arrivals. 

■'■The  need  for  such  assumptions  became  clear  through 
suggestions  from  Professor  J.  Goguen. 
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Lemma  3-1  expresses  a  request1 s  interval  rank  over  an 
interval  of  length  e  in  terms  of  interval  ranks  over  two  intervals 
whose  lengths  sum  to  e  . 

LEMMA  3-1.  For  request  j  in  request  class  i,  \J  t  >  0 , 

V<$  such  that  0<6<e, 

P(j  ,t,e)i?(j  ,t,e)  =  P  C  j  ,t,  6)  P  C  j  ,t ,  <5) 

+  [  1— P  ( j  ,t,<$)  ]  P  ( j  ,  t+6  ,  e-6)P(  j  ,t+6,e-6)  . 

PROOF.  This  lemma  is  a  consequence  of  definitions  and 
equation  manipulation.  By  definition. 


P( j ,t,e)  =  /  9  G. (x , t ) 

_  A  _L 


and 


P(j,t,e)  = 


/  x9xGiCx,t}  +  e/  9xGi(x,t) 


k  j  /  9xGi(x,t) 


Therefore , 

P(j,t,e)P(j,t,e)  =  xdxG^ix,t)  +  ef  9xGi(x,t)j, 


1  f  6  e 

=  /  x9xGi(x,t)  +  /  xdxG±tx,t) 

j  o  6 


CO  00  “1 

+  6/  9xGiCx,t)  +  (e-6)/  dxGiCx,t)  , 
e  e  J 


1_ 

k  . 
D 


I  Cj,t,6)  +  /  (x-6)  9  G .  (x,  t)  +  (e-6)  /  9  G. (x,t) 

6-X-  _L  -L 

e 
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By  change  of  variable,  y=x-6 , 


P(j,t,e)P(j,t,e)  =  £_|j(j,t,6)  +  ^  I^TET  dyFi(y+6  +  t) 

.00  “1 

+  l£'6>  Vi(y+5+t)J 


1  \t,i  „  ,  1-Fi  (t+6 )  ^ 

krL  3'  i-f±  ct)  j 


0  1-F.Jt+S)  dyFi  Cy+6  +  t) 


+  U"a)  ^£.6I^7(t+ST  dyFi(y+5+t)] 


6)  +  /  3xGi(x,t) 

6 


£-6 

/  y3yGi(y,t+6) 

00 

+  (e-6)  /  3  Gj_  (y , t+6) 


£-6 


=  UAj)[-Z‘(j,t,<5)  +  [l-P(j,t,6)]X(j,t+«,e-S)J, 

=  P( j ,t,6)P(j ,t,6)  +  [1-P  C j  ,  t ,6 )  ] P C j , t+6 , £-6 ) P  ( j  ,t+6  ,£-6)  . 


Q  .  E  .  D . 

Lemma  3-2  shows  that  over  the  period  of  its  rank  quantum, 
a  request's  rank  is  non-increasing  as  it  receives  service. 

LEMMA  3-2.  If  P(j,t)  =  p  (j  ,  t ,  £ )  (that  is,  Q(j,t)  >  £) 

then  P(j,t+6,e-6)  <  P(j,t)  \/ 

PROOF.  By  definition  of  rank,  P(j,t)  =  P(j,t,£)  <  F(j,t,6) 

6  such  that  0<6<e.  By  Lemma  3-1, 


6  such  that  0<6<£  and  G^(6,t)  <  1. 


P  ( j  ,t ,  £)  P  ( j  ,t ,  £)  =  P  C j  , t ,  6 )  P  (j  ,  t ,  6) 


+  [1-P  (j  ,  t ,  6 )  J  P  ( j  ,  t+6  ,£-6)P(j,t+6,e-6) 
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e  6 

/  3  G.  (x,t)  R  (j  ,t  ,£)  =  /  3  G.Cx,t)  i?  (  j  ,  t ,  6 ) 
o  o  x  x 

00  £ 

/  dFi(s)  /  d^Fi  (y+t) 

+  “ ^ -  x  "4 -  i?( j,t+6,e-6)  , 

/  dF-  Cs)  /  dF.  Cs) 

t  t+6  1 

r  6  £  -j  6 

/  8xGi(x,t)  +  /  8xGiCx,t)  Ii?(.j,t,e)  =  /  3^GiCx/t) 

L  o  6  J  0 

£ 

+  /  3xGiCx,t)  R (j ,t+6 ,e-8) , 

6 

6  £ 

/  8G.  (x,t)  [i?(j,t,£)-i?(j,t,6)]  =  /  9  G.  (x,t)  [f?(  j  ,t+6  ,£-6) -/?( j  ,t,E)  ]  . 
o  6  x  1 

Since  i?(j,t,£)  <  R  Cj,t,6),  t±ie  left  side  is  nonpositive,  therefore 

the  right  side  must  also  be  nonpositive.  When  G^(<$,t)  <  1, 

00  £ 

I  3  G  •  (x ,  t)  >0.  If  /  3  Gj_(x,t)  =  0,  then  i?Cj,t,6)  <  i?Cjft,£), 

6  6 

£ 

and  Q( j,t)  ^  £.  Therefore,  J  3  G.  Cx,t)  >  0.  Thus,  in  order  for 

6A  — 

the  righthand  side  to  be  nonpositive,  it  must  be  true  that 
R(  j  ,t+6  ,£-6)  <  R  Cj,t,£)  =  ^Cj/t).  Q.E.D. 

To  facilitate  comparative  evaluation,  we  need  a  means  of 
representing  any  particular  scheduling  strategy.  This  purpose  is 
served  by  the  concept  of  schedule  trees.  Knowing  initially  only 
the  distributions  from  which  service  times  are  drawn,  contingency 
plans  must  be  built  into  any  schedule  representation  for  taking 
advantage  of  that  knowledge  which  is  acquired  dynamically;  namely, 
the  actual  service  times,  which  are  learned  only  when  the  requests 
are  completed.  A  schedule  is  determined  by  specifying  for  each 
interval  of  processor  time,  for  any  system  state  that  might  exist 
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at  that  time,  the  request  to  which  the  processor  is  to  be  assigned. 
During  each  interval  of  processor  assignment,  system  state  changes 
in  one  of  two  ways;  either  the  assigned  request  completes  during 
the  interval,  or  it  fails  to  complete  while  increasing  its 
attained  service  time  by  the  duration  of  the  interval.  The 
existence  of  two  possible  state  changes  at  each  interval  leads  to 
a  representation  of  schedules  as  binary  trees.  Each  node  is  a 
pair,  (j,k) ,  where  j  is  a  request  number  and  k  is  the  number  of 
basic  units  for  which  the  processor  is  to  be  assigned  to  j .  One 
subtree  under  the  node  represents  the  schedule  to  be  followed  if 
request  j  completes  during  the  assigned  k  units.  (Request  j  must 
not  be  assigned  anywhere  in  that  subtree.)  The  second  subtree 
represents  the  schedule  to  be  followed  if  request  j  does  not  com¬ 
plete  in  the  assigned  k  units,  but  merely  increases  its  attained 
service  by  k  units. 

From  any  node  of  the  schedule  tree,  we  define  the  no- com¬ 
pletion  path  to  be  that  path  which  is  followed  down  the  schedule 
tree  until  the  next  completion  occurs.  A  single  request  may  be 
assigned  at  successive  nodes  of  the  no-completion  path  if  the 
schedule  to  be  followed  upon  completion  depends  on  when  the 
request  completes  during  the  successive  intervals.  We  need  not 
consider  inserted  idle  time  in  seeking  an  optimal  schedule  since 
any  schedule  with  inserted  idle  time  is  dominated  by  the  same 
schedule  with  inserted  idle  time  removed  as  long  as  loss  functions 
are  monotone  non-decreasing.  Because  all  distributions  are 
finite  and  all  interval  durations  are  integer  multiples  of  a  basic 
unit,  there  are  a  finite  number  of  distinct  schedules  for  com- 
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pleting  a  finite  set  of  requests.  Every  no-completion  path  is 
finite  because  eventually,  some  request's  attained  service  time 
will  reach  the  maximum  length  for  requests  of  its  class.  A 
schedule  for  completing  m  requests  is  said  to  be  of  degree  m,  and 
is  denoted  by  S^.  Appendix  II  shows  examples  of  schedule  trees 
and  how  they  are  used. 

This  main  lemma  establishes  a  necessary  condition  for  a 
schedule  to  be  optimal  with  respect  to  minimizing  expected  total 
losses . 

LEMMA  3-3.  A  schedule  can  be  optimal  only  if  it  assigns  a 
request  of  current  smallest  rank  first  for  at  least  its  rank 
quantum . 

PROOF.  The  proof  is  by  induction  on  the  degree  of  the 
schedule.  An  optimal  one  request  schedule  minimizes  loss  by 
assigning  the  only  request  to  run  to  completion.  Since  the  only 
request  is  (trivially)  the  request  of  current  smallest  rank,  and 
running  to  completion  is  assignment  for  at  least  its  rank  quantum, 
the  lemma  holds  for  schedules  of  degree  1. 

Assume  the  lemma  is  true  for  schedules  of  degree  k-1 .  If 
we  can  show  that  any  degree  k  schedule  which  does  not  run  a 
request  of  current  smallest  rank  first  for  at  least  its  rank 
quantum  can  be  improved  and  is  therefore  not  optimal,  then  the 
lemma  is  proven. 

At  time  t,  the  request  of  current  smallest  rank,  j, 

[tf (x,bx (t) ) ] . 


satisfies 


-  ,  .  ,  , , . .  min 

R  ^  'bj  ^  ^  “  X€  J  (t) 
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In  the  schedule  tree,  the  no-completion  path  is  of  particu¬ 
lar  interest  since  any  subschedule  off  the  no-ccmpletion  path  is 
of  degree  less  than  k  and  we  can  judge  its  optimality  by  the 
inductive  assumption.  As  stated  previously,  S^'s  no-completion 
path  consists  of  a  finite  number  of  intervals.  We  will  refer  to 
any  sequence  of  adjacent  intervals  along  the  no-completion  path 
which  are  assigned  to  a  single  request  as  a  segment.  We  will  say 
that  a  schedule  starts  SR  if  and  only  if  it  assigns  the  request  of 
current  smallest  rank  first  for  at  least  its  rank  quantum. 

Consider  a  schedule,  S^,  which  does  not  start  SR.  If  any 
subschedule  off  the  no-completion  path  is  not  optimal,  then  is 
not  optimal  because  with  some  probability  the  non-optimal  sub¬ 
schedule  will  be  reached,  and  an  improvement  to  the  non-optimal 
subschedule  will  reduce  the  expected  total  losses  under  S^. 
Alternatively,  if  all  subschedules  off  the  no-completion  path  are 
optimal  then,  according  to  the  inductive  hypothesis,  each  starts 


SR. 


From  all  the  subschedules  along  S  's  no-completion  path, 

K 

pick  the  latest  (that  is,  the  one  corresponding  to  the  smallest 
subtree)  which  does  not  start  SR.  We  know  that  at  least  one  such 
subschedule  exists  in  S^'s  no-completion  path  as  follows.  We 
picked  S  because  either  Cease  1)  it  does  not  assign  a  request  of 
current  smallest  rank  first,  or  Cease  2)  it  assigns  a  request  of 
current  smallest  rank  first  but  for  less  than  its  rank  quantum. 

In  case  1,  S^_  itself  does  not  start  SR.  With  respect  to  case  2, 
Lemma  3-2  shows  that  a  request  of  smallest  initial  rank  can  have 
no  larger  rank  after  being  run  for  less  than  its  rank  quantum. 
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Therefore  the  subschedule  beginning  with  the  second  segment  along 
S^'s  no-completion  path  does  not  start  SR  since  the  request 
assigned  first  by  must  still  have  smallest  current  rank. 

Call  the  selected  subschedule  S',  and  call  the  request 

K 

assigned  in  its  first  segment  i.  If  the  no-completion  path  con¬ 
tains  a  second  segment,  it  must  start  SR  by  our  method  of  selec¬ 
tion  of  S£.  If  it  starts  SR  then  it  assigns  in  the  second  seg¬ 
ment  of  the  no-completion  path  of  S£  the  request,  j,  which  is  a 

request  of  current  smallest  rank  at  the  start  of  S£. 

As  seen  earlier,  the  optimality  of  the  subschedules  entered 
upon  a  completion  in  the  first  segment  of  S£  indicates  that  they 
also  assign  j  first  since  they  start  SR.  Let  q  denote  the  dura¬ 
tion  of  the  first  segment  of  S£.  Then  any  combination  of  actual 

service  times  which  leads  to  the  use  of  S£  will  create  a  processor 

assignment  function  in  which 


Ws)  = 


(  1  **  <  s  "  ^i 

(  j  ts+x±  <  s  <  t  +x± 


+x  . 
D 


whe 


re  xi=min [ti-bi (ts) ,q] ,  x j=min [t ^-b j (ts) ,e] ,  e=Q ( j ,b ^ (tg) ) , 
and  tg  is  the  time  at  which  S£  is  reached.  Note  that  the  quan¬ 
tities  x.  and  x.  determine  uniquely  the  subschedule  to  be  fol- 
i  : 

lowed  after  t  +x.+x.  Consider  a  modified  approach  called  S/mod, 

S  1  J  •  .K 

in  which 


Zs£mod(s)  “ 


3 

i 


t  <  s  <  t_+x . 

S  s  J 

t  +x .  <  s  <  t  +x . +x . . 
s  3  s  1  j 


Again,  at  time  t„+x-+x^,  the  quantities  x.  and  x-  are  known  and 

o  J-  j  l  j 

may  be  used  to  assure  that  the  same  subschedule  is  followed 
thereafter  as  in  the  unmodified  approach.  Then  only  the  comple- 
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tion  times  of  requests  i  and  j  can  possibly  change  under  the 
modified  approach.  Therefore,  in  comparing  the  expected  total 
losses  under  the  two  strategies,  we  need  only  consider  the  losses 
accrued  by  i  and  j  during  the  interval  t  to  t  +x-+x-  since  all 
other  losses  will  be  unchanged: 


Lg£  =  (ki+kj) I (i ,b±  (ts)  ,q) 

+  [  kj  +  ki  [1-P  (i  ,bi  (tg)  ,q)  ]]  J  ( j  ,b  j  (ts)  ,  e)  , 

LS]^mod  =  (Vkj)r(j'bj<ts),e) 

+  [  kf  +  kj  [1-P( j,bj (tg)  ,e)  ]]  J(i,bi(tg)  ,q)  . 

Therefore  the  change  in  expected  total  loss  in  modifying  is 

LS*mod  ‘  LSi  m  kiP(i.bi,q)J(j.bjfe)  -  Y(i.bj,e)I(i,b.,q) 
where  b^  and  b.  are  abbreviations  for  b^(tg)  and  b^ (tg)  respec¬ 
tively.  Then 


•  LsiJ-  =  kiV(i'bi'q)p(j'bi'e) 


I  ( j  ,b j  ,e)  _  J  (i  ,bj  ,q) 

k jP ( j ,b j ,e)  k.P(i,b.,q) 


S£mod  S&  "i":  - 1'-1'  *•"'  J 

=  kikjP (i ,bi ,q) P  (j ,bj ,e)  [P  ( j ,bj ,e)  -  P(i,bi,q)]. 

The  way  we  selected  guarantees  that  P  ( j  ,b  .  ,e)  <P  (i  ,b  •  ,q)  . 

k  31 

Therefore  the  modification  to  S£  reduces  expected  total  loss 
during  S£.  This  proves  that  itself  is  not  optimal,  and  be¬ 
cause  S £  will  be  reached  with  some  probability,  as  a  whole  is 
not  optimal.  Thus  the  inductive  hypothesis  and  the  fact  that  S ^ 
does  not  start  SR  lead  inevitably  to  the  fact  that  is  not 
optimal . 

Therefore  can  be  optimal  only  if  it  assigns  a  request  of 
current  smallest  rank  first  for  at  least  its  rank  quantum.  This 
completes  the  inductive  step.  Q.E.D. 
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Consider  the  "smallest  rank"  (SR)  selection  discipline, 
which,  at  every  instant,  assigns  the  processor  to  a  request  of 
current  smallest  rank.  An  SR  schedule  is  one  derived  using  the 
SR  selection  discipline. 

THEOREM  3-1.  The  selection  rule  SR  is  the  optimal  sched¬ 
uling  discipline  with  respect  to  minimizing  expected  total  losses 
under  free  preemption,  linear  losses,  and  no  arrivals. 

PROOF.  Lemma  3-3  showed  that  no  schedule  which  is  not  SR 
can  be  optimal.  By  the  same  calculations  done  in  the  proof  of 
Lemma  3-3,  it  is  evident  that  interchanging  segments  of  equal 
current  interval  rank  does  not  change  expected  loss.  Since  ex¬ 
pected  loss  is  independent  of  selection  among  requests  of  equal 
current  rank,  all  SR  schedules  for  a  particular  finite  set  of 
requests  lead  to  the  same  expected  total  loss.  Since  the  assump¬ 
tions  of  finite  distributions  and  a  basic  unit  of  assignment 
guarantee  a  finite  number  of  possible  schedules,  and  no  non-SR 
schedules  can  be  optimal,  then,  by  default,  SR  schedules  all 
yield  minimum  expected  total  loss.  Thus  SR  is  the  optimal 
scheduling  discipline  when  losses  are  linear,  preemption  is  free, 
and  there  are  no  arrivals.  Q.E.D. 

Removal  of  the  finiteness  assumptions 

The  preceding  theorem  proof  uses  two  assumptions  of  finite¬ 
ness: 

1.  All  service  time  distributions  are  finite. 

2.  All  intervals  of  processor  assignment  have  lengths 
which  are  integer  multiples  of  a  basic  unit. 
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Because  the  real-world  processors  of  interest  do  have  inherent 
basic  cycle  times,  and  because  practical  upper  bounds  can  be 
placed  on  the  running  times  of  any  real-world  class  of  requests, 
the  assumptions  do  not  restrict  the  theorem's  applicability. 
However,  the  assumptions  are  sufficiently  unesthetic  that  we 
consider  here  how  the  need  for  them  might  be  removed. 

First,  because  there  are  distributions  which  contain  in¬ 
tervals  over  which  rank  is  monotone  increasing,  we  must  add  the 
concept  of  processor  sharing  among  requests  of  equal  current  rank. 
Then  schedule  trees  can  no  longer  be  binary  because  any  interval 
involving  processor  sharing  must  be  followed  by  a  completion 
schedule  for  each  of  the  requests  among  which  the  processor  is 
being  shared  since  any  one  might  complete  first.  If  n  requests 
are  sharing  the  processor,  the  proportion  of  the  processor's 
attention  received  by  request  k  at  time  t  is 


Second,  we  must  in  some  way  show  that,  despite  an  infinite  number 
of  possible  schedules,  the  function  of  expected  total  loss  in 
fact  attains  its  minimum  over  the  domain  of  all  possible  schedules. 
This,  together  with  the  facts  that  every  schedule  is  either  SR  or 
non-SR  and  a  non-SR  schedule  cannot  be  optimal,  is  sufficient  to 
guarantee  the  optimality  of  SR  schedules  without  the  assumptions 
of  finiteness.  One  possible  method  of  proving  that  the  expected 
total  loss  function  attains  its  minimum  is  to  define  a  topology 
on  the  set  of  all  schedule  trees,  prove  that  the  set  of  schedule 
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trees  is  compact,  and  prove  that  expected  total  loss  is  a  contin¬ 
uous  function  over  the  domain  of  schedule  trees . 

Implications  of  SR's  optimality 

Before  closing  this  chapter,  we  will  examine  the  impli¬ 
cations  of  SR's  optimality  on  some  situations  that  have  been 
extensively  investigated  in  queueing  theory  literature.  Sched¬ 
uling  strategies  RR  and  FBn ,  both  quantum  controlled,  and  their 
processor  sharing  counterparts,  PS  and  PPS ,  have  been  lauded  in 
connection  with  both  empirical  and  analytic  examinations.  As 
observed  earlier,  SR  specializes  to  PPS  in  the  special  case  of 
one  request  class  with  a  service  time  distribution  function 
having  the  property  of  monotone  increasing  rank.  However,  SR  does 
not  specialize  to  PS  or  RR  under  linear  loss  functions  except  in 
the  degenerate  case  of  exponentially  distributed  service  times, 
where  any  schedule  is  optimal.  Later,  in  Chapter  VI,  the  poten¬ 
tial  optimality  of  RR  or  PS  under  other  than  linear  loss  functions 
will  be  discussed.  Empirically,  RR  and  FBn  have  performed  favor¬ 
ably  because  no  scheduling  rules  better  suited  to  typical  request 
service  time  distributions  of  time-sharing  computer  systems  have 
been  implemented. 

As  mentioned  in  Chapter  II,  service  time  distributions  have 
frequently  been  approximated  by  exponential,  hyperexponential,  and 
log-normal  distributions,  among  others.  A  significant  common 
property  of  these  distributions  is  their  infinite  tails.  Only 
these  infinite  tails  allow  a  distribution's  rank  to  be  increasing 
for  arbitrarily  large  attained  service  times.  Let  us  reexamine 
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the  procedure  of  approximating  the  service  time  distribution  in  a 
particular  request  class.  Because  the  mean  is  frequently  an 
order  of  magnitude  larger  than  the  median  and  the  coefficient  of 
variation  is  larger  than  one,  a  distribution  closely  resembling  a 
hyperexponential  is  indicated.  However  the  conclusion  must  be 
tempered  for  two  reasons;  first  such  an  approximation  is  certainly 
much  more  valid  near  the  origin  than  in  the  distribution  tail, 
where  the  sample  points  are  relatively  sparse,  and  second,  service 
time  distributions  are  in  fact  finite  for  practical  reasons.  With 
these  added  considerations,  we  question  whether  approximation  of 
distributions  by  a  truncated  hyperexponential, 
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is  not  equally  valid  for  appropriately  large  M.  The  exponential, 
similarly  truncated,  acquires  the  property  of  decreasing  rank  so 
that  under  linear  losses,  FIFO  is  the  optimal  scheduling  rule. 

The  truncated  hyperexponential  has  increasing  rank  to  a  point, 
then  decreasing  rank.  In  this  case,  SR  scheduling  indicates  PPS 
to  be  appropriate  for  scheduling  requests  which  are  in  their  early 
stages,  but  past  a  certain  point  of  attained  service,  preemption 
should  be  employed  only  upon  the  arrival  of  a  request  with  initial 
smaller  rank.  The  ratio  of  the  lengths  of  the  intervals  of 
increasing  rank  and  decreasing  rank  depends  on  the  cumulative 
percentile  of  the  hyperexponential  distribution  at  which  it  is 
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truncated,  and  how  close  to  exponential  the  distribution  is. 
Truncation  even  at  high  percentiles  will  create  very  significant 
intervals  of  decreasing  rank  in  hyperexponential  distributions 
which  are  close  to  exponential. 

The  existence  of  an  important  degree  of  freedom  in  SR 
scheduling  should  be  mentioned.  We  have  stated  that  most  empir¬ 
ical  studies  have  shown  service  time  distributions  to  have 
disturbingly  high  variance.  This  fact  minimizes  the  utility  in 
scheduling  of  knowing  the  mean  service  time.  However,  if  any 
distinguishing  information  accompanies  a  request,  a  possibility 
exists  of  partitioning  the  requests  into  request  classes  such  that 
the  service  time  distributions  by  class  will  have  much  lower 
coefficients  of  variation. 


CHAPTER  IV 


SCHEDULING  UNDER  ARRIVALS 


The  Impact  of  arriving  requests 

The  most  restrictive  assumption  made  in  Chapters  I,  II,  and 
III  was  that  all  requests  were  initially  available  for  service. 

A  more  interesting  situation  results  when  the  arrival  of  new 
requests  is  allowed.  As  was  pointed  out  earlier,  an  arrival 
process  may  be  characterized  at  one  of  several  levels  of  complex¬ 
ity.  Random,  or  Poisson,  arrivals  result  when  interarrival  times 
are  exponentially  distributed.  Independent  arrivals  occur  when 
interarrival  times  are  independent  samples  from  any  distribution. 
Under  arbitrary  arrivals,  we  must  allow  the  possibility  of  depen¬ 
dency  among  successive  interarrival  times.  We  will  assume 
throughout,  however,  that  arrivals  are  independent  of  the  service 
discipline  used  in  assigning  the  processor. 

Poisson  distributed  arrivals  are  frequently  termed  "random" 
because  the  arrivals  in  various  time  intervals  of  the  same  length 
are  uniformly  distributed  within  the  interval.  Many  significant 
results  of  queueing  theory  are  based  on  the  assumption  of  a 
Poisson  arrival  process. 
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In  many  cases ,  scheduling  rules  known  to  be  optimal  when 
there  are  no  arrivals  have  been  shown  optimal  also  with  the  addi¬ 
tion  of  Poisson  arrivals.  Among  these  are  SPT/c  with  preemption 
disallowed  and  service  times  known  exactly,  and  SEPT/c  with  pre¬ 
emption  disallowed  and  service  times  known  probabilistically. 
However,  under  more  general  arrival  processes,  SPT/c  and  SEPT/c 
are  not  necessarily  optimal.  When  preemption  is  allowed,  SRPT  is 
optimal  under  arbitrary  arrival  patterns. 

The  Rule  SKPT/c  under  arrivals 

Another  rule,  SRPT/c,  which  is  closely  related  to  SRPT,  is 
also  of  interest.  The  rule  SR  specializes  to  SRPT/c  when  service 
times  are  known  exactly.  Thus  SRPT/c  is  known  optimal  under  no 
arrivals  (or  after  the  last  arrival  of  a  busy  period) .  Encouraged 
by  this  and  by  SRPT's  optimality  under  arbitrary  arrivals,  we 
might  conjecture  that  SRPT/c  remains  optimal  under  arrivals.  How¬ 
ever,  for  arbitrary  arrival  patterns,  the  following  is  a  counter¬ 
example  to  SRPT/c' s  optimality. 

Requests  and  j 2  are  currently  in  system.  Request  j  has 
loss  rate  k^  and  remaining  service  time  t^,  and  has  loss  rate 
k^  and  remaining  service  time  t^  where  t^/k-^  <  t2/k2  although 
^2  <  Assume  that  the  arrival  process  indicates  that  with 

probability  p,  another  request,  j^,  with  service  time  t^  and  loss 
rate  k^  will  arrive  x  time  units  in  the  future  where  t2  <  x  <  t^. 
No  other  arrivals  will  occur  in  the  next  ti+t2+t3  units  of  time. 

We  wish  to  determine  whether  there  are  values  for  these  param¬ 
eters  such  that  the  expected  total  cost  is  minimized  by  assigning 
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j  first  although  SRPT/c  would  select  .  We  will  use  the  fact 
that  SRPT/c  is  optimal  after  the  last  arrival  of  a  busy  period. 

If  t3/k^<  (t1~x)/k1  <  t2/k2 ,  Figure  6  shows  our  alternatives  and 
their  consequences.  The  expected  costs  of  finishing  the  busy 
period  under  our  two  options  are 

Lj2-first  =  (kl+k2>t2  +  kltl  +  P(k1+k3)t3, 
Lj!-first  =  <k1+k2)tl  +  k2fc2  +  P (k1+k2+k3)  t3 . 

Thus  j2  should  be  assigned  first  if  and  only  if 

(k1+k2)t1  +  k2t2  +  P (k1+k2+k3) t3  <  (k1+k2) t2  +  k1t1  +  p(k1+k3)t3, 
or 


tl  Pfc3 


If  pt^  >  (k^t2/k2)  -  t^,  we  have  a  situation  in  which  SRPT/c  is 
not  optimal.  For  example,  we  may  set  the  variables  as  follows: 

k  ^  =  8  ,  k2  =  3  ,  p  =  . 5 ,  k  3  =  2  0  , 

t-^  =  16 ,  t2  =  8 ,  x  =  9,  t3  =  14. 

This  example  also  shows  that  SR  is  not  always  optimal  under 
arbitrary  arrivals  since  SR  schedules  identically  to  SRPT/c  when 
service  times  are  known  exactly.  By  careful  construction,  the 
counterexample  exists  even  in  the  more  restricted  situation  of 
independent  arrivals.  The  interarrival  distributions  for  class 
types  1  and  2  would  consist  only  of  long  intervals,  while  the 
distribution  for  class  3  would  be  of  the  two-spike  variety  with 
intervals  either  short  C-x)  or  long  (>t-^+t2+t3)  . 

The  Extent  of  SRPT/c 's  optimality 


Only  under  Poisson  arrivals  does  SRPT/c' s  optimality  remain 
uncertain.  Schrage  has  suggested  a  simple  necessary  condition  to 
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(B)  Assigning  first,  without  and  with  an  arrival. 


Figure  6. — Consequences  of  two  strategies  under  general 
arrivals . 
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test  SRPT/c  in  this  case.  The  rule  SRPT/c  can  be  optimal  glob¬ 
ally  with  respect  to  minimizing  expected  rate  of  total  loss  only 
if  it  is  optimal  within  every  family  of  rules  to  which  it  belongs. 
In  a  two  request  class  system,  class  1  requests  have  service  time 
tj  and  loss  rate  k-^  while  class  2  requests  have  service  time  t^ 
and  loss  rate  k2  .  Assume  that  t^/k^  <  t2/k2*  Consider  the  fol¬ 
lowing  parameterized  family  of  scheduling  rules  for  the  two  class 
system:  Class  1  requests  have  preemptive-resume  priority  over 

class  2  requests  whose  remaining  service  time  is  greater  than  0, 
and  non-preemptive  priority  otherwise.  For  various  values  of  the 
scheduling  parameter  0,  a  family  of  scheduling  strategies  is  de¬ 
fined.  Setting  0  to  zero  leads  to  a  standard  preemptive-resume 
priority  system,  setting  0  to  k2t-^/k-^  leads  to  SRPT/c  scheduling, 
and  0  >  t^  leads  to  a  non-preemptive  priority  system.  We  will 
determine  the  setting  of  0  which  minimizes  expected  rate  of  total 
loss.  If  the  optimal  value  of  0,  0  is  not  such  that  t^/k-^  = 

0Opt/^2'  then  SRPT/c  is  not  optimal  within  the  parameterized  fam¬ 
ily  of  rules,  and  therefore  cannot  be  globally  optimal. 

For  i  =  1,  2,  let 

=  arrival  rate  of  class  i  requests  (interarrival 
times  are  exponentially  distributed  with 
parameter  , 

=  expected  waiting  time  of  class  i  requests, 

R^  =  expected  residence  time  of  class  i  requests 
(time  from  start  to  completion  of  service) , 


'Private  communication  in  May,  1971. 
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and  =  expected  processing  time  for  class  i  requests. 

The  expression  for  expected  rate  of  total  loss,  which  we  wish  to 
minimize,  is  then 


L(0)  =  X^k^[W^+R-^]  +  ^2k2^2+^2^  • 

To  determine  the  ' s  and  1 s  in  terms  of  0,  we  may  use  two 
queueing  theory  results  {18] : 

1.  For  k-level  non-preemptive  priority  disciplines. 
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2.  For  preemptive-resume  systems, 


R. 

l 


P  . 

l 


i-1 

i  -  y  a  .p . 


=1  :  i 


j=l 


In  the  particular  system  of  interest  here,  we  have 


since  class  1  requests  cannot  be  preempted, 
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since  class  2  requests  are  preemptible  only  during  their  first 


t2  -  0  units  of  service. 


2  2 
^1^1  +  ^2^2 

W2  =  2 (1-A1t”-A2t2) (1-A 1t1) 


since  class  2  requests  must  wait  for  all  requests  already  in 
system  and  any  arriving  class  1  requests. 
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and 
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since  class  2  requests  are  invisible  to  class  1  requests  except 
in  their  last  0  units  of  processing. 


Substituting  into  the  objective  function  yields 


L(0)  = 


hki 


.  ,  Xltl2+X202 

1  2(1-X1t1) 
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The  first  derivative  with  respect  to  0  is 
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The  second  derivative  is 
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In  an  unsaturated  system,  X^t^  <  1,  so  the  second  derivative  is 
positive.  When  the  second  derivative  is  positive,  cost  is  mini¬ 
mized  where  the  first  derivative  is  zero.  Therefore  minimum 
cost  requires  that 


or 


kn  0 
1  opt 


tlk2 


=  0, 


Thus  the  optimal  strategy  within  the  parameterized  class  consid¬ 
ered  is  equivalent  to  SRPT/c. 

The  counterexample  to  SRPT/c' s  optimality  under  general 
arrivals  came  about  because  a  short  request  could  be  completed 
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before  the  next  arrival,  but  a  longer  request  was  unlikely  to  fin¬ 
ish  before  being  preempted  by  the  arrival  of  a  more  critical 
request.  Although  the  longer  request  had  lower  ratio  of  service 
time  to  loss  rate,  the  potential  of  an  arrival  made  the  risk  of 
assigning  it  initially  unworthwhile .  Under  Poisson  arrivals,  the 
expected  time  to  the  arrival  of  a  new  request  of  any  class  is 
always  the  same.  There  is  no  motivation  to  delay  important  work 
to  await  a  more  favorable  situation  of  anticipated  arrivals. 

Thus,  intuitively,  it  seems  that  SRPT/c's  weakness  as  reflected 
in  the  counterexample  presents  no  difficulty  when  arrivals  are 
Poisson . 

We  will  now  argue  that  a  selection  rule  is  sufficient  to 
minimize  expected  rate  of  total  loss  under  Poisson  arrivals,  and 
then  prove  that  among  selection  rules,  SRPT/c  is  optimal.  Later 
it  will  be  shown  that  similar  arguments  hold  for  SR  scheduling 
under  Poisson  arrivals  when  only  probabilistic  information  about 
service  times  is  available.  Recall  that  a  selection  rule  ranks 
requests  for  assignment  by  increasing  (by  convention)  values  of  a 
function  of  the  attributes  of  each  individual  request. 

CONJECTURE  4-1.  Under  Poisson  arrivals,  linear  losses  and 
known  service  times,  a  selection  rule  suffices  to  minimize  the 
expected  rate  of  total  loss. 

ARGUMENT.  For  each  request  class,  loss  rate,  service  time, 
and  rate  of  new  arrivals  are  known.  Although  the  times  of  pre¬ 
vious  arrivals  may  be  known,  the  properties  of  a  Poisson  arrival 
process  make  future  arrival  times  independent  of  previous  arrival 
times.  Thus  the  outlook  of  anticipated  arrivals  is  constant  over 
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time.  The  total  loss  accrued  by  any  request  depends  only  on  when 
it  is  completed,  not  on  the  order  in  which  any  other  requests 
are  assigned.  Therefore,  interchanging  the  order  of  serving  por¬ 
tions  of  two  requests  affects  only  the  costs  accrued  by  those  two 
requests.  Thus  the  choice  of  which  of  two  requests  should  be 
served  first  is  independent  of  what  other  requests  are  currently 
in  system.  This  implies  transitivity  of  the  preference  ordering. 
Further  the  choice  between  any  two  requests  is  always  the  same 
since  the  anticipated  arrival  pattern  does  not  change  with  time. 
Thus  the  selection  between  any  two  requests  is  based  only  on  the 
attributes  of  those  particular  requests.  Therefore  a  selection 
rule  which  uses  a  function  of  the  service  time,  loss  rate,  and 
attained  service  of  each  request  is  sufficient  to  minimize  the 
expected  rate  of  total  loss. 

LEMMA  4-2.  Under  linear  losses  and  Poisson  arrivals,  the 
selection  rule  which  minimizes  the  expected  rate  of  total  loss 
uses  a  selection  function,  f(t£,k^,b£) ,  where  the  arguments  of 
the  function  are  service  time,  loss  rate,  and  attained  service  of 
request  i.  The  optimal  rule  assigns  the  processor  to  that  re¬ 
quest,  j,  such  that 


The  selection  function  has  the  following  properties: 


(i)  t.-b. 

1  i 


==>  f(ti,k,bi)  <  f(tj,k,bj) 
f (t,ki,b)  <  f (t,kj ,b) , 
f  (t,k,bj_)  <  f  (t  ,k,bj)  . 


<  t.-b 
D  j 


(ii)  k .  >  k  . 

i  D 


(iii)  bi  >  bj 


PROOF.  (i)  Because  SRPT  is  known  optimal  for  arbitrary 
arrivals  when  loss  rates  are  identical,  the  optimal  selection 
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rule  under  Poisson  arrivals  must  schedule  identically  to  SEPT  in 
the  special  case  of  identical  loss  rates.  Therefore 

ti-bi  <  tj-bj  ==>  f  (tj_  ,k,bj_)  <  f(tj,k,bj). 

(ii)  For  any  requests,  i  and  j,  with  remaining  service 
times  t^  and  t  ,  with  kj  fixed,  k^  may  be  increased  enough  to  make 
it  desirable  to  assign  request  i  before  request  j.  This  indicates 
that  increasing  k^  makes  request  i  more  desirable  to  schedule 
early.  Therefore, 

ki  >  kj  ==>  f(t,ki,b)  <  f  (t ,k_.  ,b)  . 

(iii)  If  b^  >  bj  then  t-b^  <  t-bj.  By  part  CD  , 

t-b^  <  t-bj  ==>  f(t,k,b.j_)  <  f(t,k,bj).  Therefore, 

bi  >  bj  ==>  f(t,k,bi)  <  f  (t,k,b j)  .  Q.E.D. 

THEOREM  4-1.  Under  linear  losses,  Poisson  arrivals,  and 
known  service  times,  SRPT/c  scheduling  minimizes  the  expected 
rate  of  total  loss  among  selection  rules . 

PROOF.  Part  (iii)  of  Lemma  4-2  indicates  that  the  selection 
function  decreases  with  increasing  attained  service.  Thus  if 
request  i  is  once  selected  over  request  j ,  it  will  be  completed 
before  request  j  receives  any  (additional)  service.  Request  i 
will  be  preempted  only  by  busy  periods  of  arrivals  with  low  ini¬ 
tial  selection  function  values.  Note  that  as  a  request  is  served, 
fewer  and  fewer  arrivals  will  have  an  initial  selection  function 
value  low  enough  to  preempt  the  request  in  service.  The  probabil¬ 
ity  of  a  preempting  arrival  and  the  expected  length  of  a  busy 
period  of  arrivals  with  low  selection  function  values  are  both 
non-increasing,  and  generally  decreasing,  functions  of  the 
attained  service  of  the  request  being  served.  Because  only 
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arrivals  motivate  preemption,  between  arrivals,  we  need  only 
consider  the  order  in  which  requests  currently  in  system  will  be 
selected.  Consider  two  orderings  which  differ  only  by  the  inter¬ 
change  of  two  requests,  i  and  j.  The  expected  residence  time  of 
either  request  from  its  initiation  to  its  completion  is  indepen¬ 
dent  of  whether  it  is  initiated  first  or  second.  Thus  the  total 
difference  in  expected  loss  that  we  must  consider  in  comparing 
the  two  orderings  is  the  loss  rate  of  the  request  initiated  sec¬ 
ond  times  the  interval  from  the  initiation  of  the  first  to  the 
initiation  of  the  second.  We  will  use  S  (t  , kv ,bv , t„ , kw ,b.r)  to 
denote  the  expected  time  from  the  initiation  of  request  x  to  the 
initiation  of  request  y  under  the  condition  that  y  is  initiated 
only  after  all  arrivals  whose  initial  selection  function  value  is 
less  than  f(t^.,k^,by)  have  been  completed. 

We  define 

p(tx'kx'bx}  =  I  Avtv  where  Y  =  (y  |  f (t  ,k  , 0) <f (t  ,k  ,bx) } , 

yeY  *  1  Y  y 

=  the  load  factor  of  requests  which  would  be 

initiated  ahead  of  request  x  with  attained 

service  b  . 

x 

Then,  because  the  arrivals  from  all  classes  are  Poisson, 


t  -  b 
x  x 


s(tx'kx-bx'ty'ky'by)  "  i  -  p(ty,ky,by) • 


The  expected  difference  in  cost  between  the  orderings  which 

differ  by  the  interchange  of  i  and  j  must  equal  zero  if  and  only 

if  their  selection  function  values  with  their  current  attained 

service  times  are  the  same.  That  is,  if  f(t.,k.,b  )=f(t  ,k  ,b  ) , 

ill  j  j  j 
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then 


or 


kj^S  (tj  ,kj  ,bj  ,ti,ki/bi)  -  kjS  (ti,ki  ,b±  ,tj  ,kj  rb j )  =  0, 


tj  -  bi  t±  -  b± 

ki  1-p  ,ki  ,bi)  ”  kj  1-p  (t  j  ,k_.  ,b_. ) 


=  0. 


Since 


f (ti ,k^  ,bi) 


f (tj ,k j  ,bj) 


<==>  pCti/ki,bi) 


p  (t  j  ,k j  ,b j )  f 


we  have 


f(t.  fkifb.)  =  f(t.,k.,b.)  <==>  k  (t.-b.)  -  k.(t.-b.)  =  0, 

j  J  J  ^  D  D  D .  * . 


or 


That  is,  the  selection  function  values  are  the  same  if  and  only 

if  the  requests  have  identical  ratios  of  remaining  service  time 

to  loss  rate.  Therefore,  f(t  ,k  ,b  )  =  (t  -b  ) /k  is  a  selection 

xxx  x  x' '  x 

function  which  minimizes  the  expected  rate  of  total  loss,  and 
SRPT/c  is  an  optimal  selection  rule  under  linear  losses  and 
Poisson  arrivals  when  service  times  are  known.  Q.E.D. 

It  should  be  possible  to  prove  SRPT/c' s  optimality  under 
Poisson  arrivals  more  directly  by  a  traditional  interchange 
argument.  However,  such  an  argument  must  take  an  inductive  form 
in  order  to  account  for  the  arriving  requests,  and  the  conditions 
of  induction  disturb  the  local  assumption  of  Poisson  arrivals, 
making  it  difficult  to  characterize  the  anticipated  arrivals. 


The  Rule  SR  under  Poisson  arrivals 

We  have  demonstrated  the  optimality  of  SRPT/c  among  selec¬ 
tion  rules  under  Poisson  arrivals  when  remaining  service  times 
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are  known  precisely.  Next  consider  the  situation  of  knowing  only 
the  distribution  of  service  times  for  each  request  class.  In 
Chapter  III,  SR  was  proven  to  be  the  optimal  scheduling  rule  with 
no  arrivals.  We  will  argue  that  this  optimality  holds  with  the 
addition  of  Poisson  arrivals. 

CONJECTURE  4-3.  Knowing  only  service  time  distributions, 
a  selection  rule  suffices  to  minimize  the  expected  rate  of  total 
loss  when  losses  are  linear  and  arrivals  are  Poisson. 

ARGUMENT.  In  proving  the  optimality  of  SR  scheduling  under 
no  arrivals,  the  scheduling  units  were  intervals  of  requests  over 
which  rank  was  monotone  non-increasing.  For  each  interval,  we 
determined  (1)  the  expected  processor  time  invested  in  assigning 
the  request  for  that  interval,  and  (2)  the  probability  that  the 
request  would  complete  within  the  interval.  Again,  under  Poisson 
arrivals,  we  may  treat  requests  as  sequences  of  such  intervals. 

The  order  of  service  may  involve  interleaving  intervals  of  various 
requests,  but  within  a  single  request,  intervals  must  be  assigned 
in  their  inherent  order.  The  order  in  which  components  of  two 
particular  requests  are  assigned  does  not  influence  the  expected 
losses  of  other  requests  in  system.  Therefore,  as  in  the  case  of 
precise  service  time  knowledge,  the  selection  between  initial 
remaining  components  of  two  requests  is  a  constant  choice  inde¬ 
pendent  of  other  requests  in  system.  Thus,  selection  between  two 
requests  depends  only  on  the  attributes  of  those  requests:  ser¬ 
vice  time  distribution,  attained  service  time,  and  loss  rate. 
Therefore,  a  selection  function  based  on  these  attributes  is 
sufficient  to  minimize  expected  rate  of  total  loss. 


81 


THEOREM  4-2.  Under  linear  losses,  Poisson  arrivals,  and 
known  service  time  distributions,  SR  scheduling  minimizes  the 
expected  rate  of  total  loss  among  selection  rules. 

PROOF.  Let  f CG  (t ,b  ) ,k  ,b  )  be  the  selection  function 

X  X  X  X 

associated  with  the  optimal  selection  rule,  where  G  (t,b  )  is  re- 

X  X 

quest  x's  conditional  remaining  service  time  distribution  with 

attained  service  b  ,  k  is  its  loss  rate,  and  b  is  its  attained 

x  x  x 

service.  As  long  as  the  system  is  not  saturated,  the  selection 
function  is  independent  of  the  rate  of  Poisson  arrivals  since  the 
anticipated  arrival  outlook  remains  constant  over  time.  But  with 
an  arrival  rate  of  zero,  SR's  optimality  under  no  arrivals  indi¬ 
cates  that 


f(G.  (t,b.)  ,k.  ,b  .  )  <f  (G  .  (t,b.)  ,k.,b.)  <==>  R  (i  ,b  .  )  <R  ( j  ,b  . )  . 

Since  the  selection  function  is  independent  of  the  arrival  rate, 
scheduling  according  to  the  Smallest  Rank  selection  rule  is  opti¬ 
mal  also  under  Poisson  arrivals  with  non-zero  arrival  rate.  Q.E.D. 


Scheduling  under  general  arrivals 

When  the  arrival  process  becomes  more  complex  than  random 
arrivals,  the  information  known  about  arrivals  makes  selection 
rules  insufficient  to  attain  optimal  scheduling.  Only  with  ser¬ 
vice  times  known,  preemption  allowed,  and  identical  linear  losses 
has  an  optimal  scheduling  rule  under  arbitrary  arrivals  been 
determined.  Even  the  apparently  simple  scheduling  problem  where 
preemption  is  disallowed  and  service  times  are  known  has  not  been 
solved . 

The  counterexample  given  earlier  showed  that  SRPT/c  failed 
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under  arbitrary  arrivals  although  SRPT  is  known  optimal  under 
arbitrary  arrivals  when  loss  rates  are  identical  for  all  requests. 
Under  arbitrary  arrivals,  the  failure  of  SRPT/c  with  non-identical 
losses  implies  the  failure  of  SR  since  SR  specializes  to  SRPT/c 
when  service  times  are  known  precisely.  However,  it  is  possible 
that  the  added  restriction  of  identical  linear  losses  might  add  to 
the  generality  of  arrival  patterns  under  which  SR  is  optimal. 

Note  that  the  counterexample  to  the  optimality  of  SRPT/c  and 
SR  depended  on  the  possibility  of  one  request  being  longer  than 
another  yet  having  a  smaller  ratio  of  service  time  to  loss  rate. 
The  added  restriction  of  identical  linear  losses  precludes  such  a 
possibility . 

However,  further  investigation  reveals  a  counterexample  to 
SR's  optimality  under  arbitrary  arrivals  even  with  the  restriction 
that  all  loss  rates  are  identical.  Consider  the  following  three 


request  class  system: 

ki  =  k2  = 

F1Ct)  = 

k3  = 

1, 

0<t<3 

3<t<10 

10<t 

(i?  C  j  x  f  0)  =6)  , 

F2(t)  = 

{ l 

0<t<  5 

5<t 

(i?(j2,0)=5) , 

u> 

» — - 

r+ 

II 

{ ? 

i — i 

V 

-P  -P 

VI  VI 

O  rH 

Ci?(j3,0)=i) . 

At  time  t=0 ,  one  class  1  request  and  one  class  2  request 
are  in  system.  It  is  known  that  the  only  arrivals  in  the  rest  of 
the  busy  period  will  be  n  class  3  requests  which  will  arrive 
between  t=3  and  t=4.  Then  SR  scheduling  yields  an  expected 
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total  cost  of 

LgR  =  .5[ (n+5)+(n+8)+CnJ  +  . 5 [ (n+5) + (n+15) +Cn] 
where  Cn  is  the  cost  accrued  by  the  n  class  3  requests.  But 
consider  assigning  the  class  1  request  for  3  time  units  ahead  of 
the  class  2  request,  while  maintaining  top  priority  service  to 
class  3  requests.  The  modified  strategy  yields  an  expected  total 
cost  of 

Lmod  =  -5[(3)+(n+8)+Cn]  +  . 5 [ (n+8) + (n+15) +Cn] . 

We  have  a  counterexample  to  SR's  optimality  if 

Lmod  <  LSR' 
or 

. 5 [n+ll+C^]  +  .  5[2n+23+C^]  <  .5[2n+13+Cn]  +  . 5[2n+20+C^] , 
which  reduces  to 

1  <  n. 

Therefore,  if  n  >  2  then  SR  is  not  optimal  under  this  situation 
of  arbitrary  arrivals  although  all  loss  rates  are  identical. 

Aside  from  the  situations  treated  above,  little  of  general 
utility  can  be  said  about  scheduling  under  general  arrivals, 
although  it  is  likely  that  the  problem  can  be  solved  for  many 
special  cases  where  the  particular  distributions  of  interarrival 
times  are  known.  Having  noted  these  difficulties  of  scheduling 
under  general  arrivals,  it  is  comforting  to  remember  the  evidence 
that  in  many  actual  scheduling  environments,  the  arrival  process 
is  best  characterized  as  Poisson  [63] . 


CHAPTER  V 


SCHEDULING  WITH  SIGNIFICANT  PREEMPTION  COST 


The  Impact  of  preemption  overhead 

In  Chapter  III,  the  assumption  that  the  amount  of  processor 
time  lost  in  preemption  is  negligible  led  us  to  the  development 
of  a  new  scheduling  strategy  which  attained  minimum  expected  total 
loss  through  judicious  use  of  preemption  based  on  dynamic  knowl¬ 
edge  of  remaining  service  time  distributions.  As  we  saw,  certain 
forms  of  service  distributions  led  to  arbitrarily  frequent  invo¬ 
cation  of  preemption.  In  this  chapter,  we  remove  the  tenuous 
assumption  that  preemption  costs  are  insignificant.  We  will  find 
that  many  principles  derived  in  Chapter  III  remain  applicable 
with  reasonable  extensions  and  modifications. 

Preemption  in  a  simple  system 

We  examine  first  a  very  simple  situation  in  which  we  have  a 
single  request  class  with  a  two-spike  distribution  of  service 
times : 

0  0  <  t  <  £ 

F(t)=  p  e  <  t  <  3 

1  3  <  t 

All  requests  have  unit  loss  rate. 
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In  Chapter  III  we  found  that  with  service  time  distributions  of 
this  form,  the  appropriate  use  of  preemption  could  reduce  expected 
cost  whenever  g>£(l+p)/p.  Let  C  denote  the  amount  of  processor 
time  required  to  preempt  one  request  and  initiate  another.  With 
C  >  0 ,  we  would  expect  preemption  to  be  less  frequently  desirable 
than  with  C  =  0  as  in  preceding  chapters.  Because  preemption  need 
only  be  considered  at  points  of  increasing  rank,  the  distribution 
function  F(t)  leads  us  to  consider  preemption  only  at  one  point. 

Rank  increases  at  point  £  if  and  only  if  g  >  sCl+p)/p.  If  initially, 
several  requests  are  available  for  processing,  we  may  characterize 
significant  successive  system  states  by  triples  of  the  form 
(u,s,t)  where 

u  =  number  of  unstarted  jobs, 

s  =  number  of  jobs  started  but  preempted  after  failing 
to  complete  in  £  units, 

and  t  =  attained  service  of  job  to  which  processor  is 

currently  assigned  (t  =  $  means  the  processor  is 
currently  unassigned) . 

Three  classes  of  system  state  are  of  interest: 

1.  At  nodes  (u,s,$)  the  processor  is  unassigned  and,  if 
u>0  and  s>0,  a  scheduling  decision  is  required  with 
regard  to  whether  to  load  an  unstarted  request  or 
one  preempted  earlier  after  attaining  £  units  of 
service . 

2.  At  nodes  (u,s,c) ,  the  assigned  request  has  attained 
£  units  of  service  without  completing,  and  a  sched¬ 
uling  decision  is  required  to  determine  whether  or 
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not  to  preempt  the  request  currently  assigned  in 
favor  of  an  unstarted  request. 

3.  At  nodes  (u,s,0) ,  the  assigned  request  may  be  either 
short  or  long.  After  e  units  of  service  the  state 
becomes  (u,s,$)  with  probability  p  and  (u,s,s)  with 
probability  1-p. 

If  3  <  e(l+p)/p,  preemption  should  never  be  done.  Optimal  sched¬ 
uling  involves  selecting  unstarted  requests  at  random  and  running 
them  to  completion. 

Assigning  an  available  processor 

For  the  remainder  of  the  analysis  of  this  system,  we  will 
assume  that  3  >  e(l+p)/p.  Figure  7  shows  the  state  transition 
diagram  of  the  system.  Note  that  the  transitions  from  states 
(u,s,0)  are  probabilistic,  and  the  transitions  from  states 
(u,s,e)  depend  on  the  choice  of  whether  to  preempt  or  continue. 

The  potential  transitions  of  preempting  an  unstarted  job,  or 
loading  a  preempted  job  rather  than  an  unstarted  one  are  not 
shown  in  the  figure  because  they  are  always  non-optimal.  We  will 
show  that  every  schedule  which  loads  a  preempted  request  in  pref¬ 
erence  to  an  unstarted  one  is  dominated,  with  respect  to  expected 
total  cost,  by  one  which  loads  an  unstarted  request.  Consider  any 
schedule,  S-pre,  which,  at  state  (u,s,$),  with  u>0,  chooses  to 
load  a  preempted  request.  Locate  where  the  last  unstarted  request 
is  finally  loaded.  The  number  in  system  then  is  k  <  (u+s-1) .  If 
it  does  not  complete  in  e,  any  schedule  which  preempts  it  is  dom¬ 
inated  by  one  which  runs  it  to  completion  because  only  requests 
with  remaining  service  time  3~£  remain,  and  preemption  yields  no 
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(4,0,$)  (3,1,$)  (2,2,$)  (1,3,$)  (0,4,$) 


(3,0,0)  (3,0,e)  (2,1,0)  (2,l,e)  (1,2,0)  (l,2,e)  (0,3,0)  (0,3,e) 


(2,0,0)  (2 , 0 ,  e)  (1,1,0)  (1,1, e)  (0,2,0)  (0,2,e) 


(2,0,$) 


U,l,$) 


(0,2,$) 


(1,0,$) 


(0,1,$) 


Figure  7. — State  transition  diagram  of  a  simple  system. 
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advantage  despite  its  costs.  But  now  consider  modifying  the 
schedule  by  interchanging  the  preempted  request  which  is  chosen 
for  loading  first  and  the  last  unstarted  request  which  is  run  to 
completion  with  k  requests  in  system.  Call  the  modified  schedule 
S-uns.  With  this  interchange,  the  expected  time  that  exactly  u+s 
jobs  are  in  system  changes  from  (3-e  to  pe+(l-p)$,  and  the  ex¬ 
pected  time  that  exactly  k<u+s-l  jobs  are  in  system  changes  from 
pe+(l-p)g  to  8-s.  The  change  in  expected  cost  brought  about  by 
the  interchange  is 


LS-uns“  Ls-pre  =  t  (u+s)  (pe+(l-p)8)  +  k(|3-e)] 

-  [  (u+s)  ((3-e)  +  k(pe+ Cl-p)  8)  3  » 
=  (u+s-k)  [pe+ (1-p) 8- (8-e) ]  / 


=  -p (u+s-k) 


Hz  £ 

p 


The  factors  (u+s-k) ,  p  and  (3-e(l+p)/p  are  all  known  to  be  positive. 
Therefore  the  interchange  has  reduced  expected  total  cost.  Be¬ 
cause  any  schedule  which  chooses  to  load  a  preempted  job  is  domi¬ 
nated  by  one  choosing  to  load  an  unstarted  job,  the  optimal  deci¬ 
sion  from  state  (u,s,$)  with  u>0  is  always  to  load  an  unstarted 
request. 


An  Optimal  preemption  strategy 

Consider  next  the  preemption  decision  necessitated  at  nodes 
(u,s,e)  for  u>0.  The  decision  at  one  node  depends  on  those  which 
may  be  encountered  thereafter.  Thus  we  must  specify  decisions 
iteratively,  allowing  successively  longer  sequences  of  decision 
points.  First  consider  the  family  of  nodes  (l,s,e).  With  just 
one  unstarted  request  remaining,  no  decision  points  beyond  this 
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one  will  be  encountered.  Compare  the  expected  costs: 


s 


LS-cont  =  ts+2)  (3-e)  +  (.s+l)  [pe+Cl-p)  3]  +  l  i(g-e) 

i=l 


s 


LS-pre  =  Cs+2)  [C+pe+ Cl-p) 6]  + Cs+1)  (6-e) +  l  i(3-e). 


i=l 


Therefore  preemption  reduces  expected  cost  if  and  only  if 


(s+2) C+ [pe+ (1-p) 3] - (3-e)  <  0 , 


or 


Solving  for  s,  which  indicates  system  state. 


s  <  ^  [p3- (1+p) e] -2 . 


(2) 


Thus  parameters  C,  p,  z,  and  3  determine  a  threshold  value  of  s. 
If  the  number  of  preempted  requests  in  system  exceeds  the  thres¬ 
hold,  preemption  is  undesirable.  A  numeric  example  follows: 


Let  p  =  .5,  z  =  1,  and  3  =  11.  Then  the  threshold  value  of 
s  is  (1/C)  [p 3—  (1+p) £] -2  =  (6/C)-2. 


Therefore,  C  =  0  ==>  always  preempt, 


C  =  -j=r  ==>  preempt  iff  s  <  10 
C  =  1  ==>  preempt  iff  s  <  4, 


C  =  2  ==>  preempt  iff  s  <  1 


and 


C  =  3  ==>  never  preempt. 


At  all  nodes  of  the  form  (0,s,e)  ,  "continue"  is  the  optimal 
decision.  We  use  this  fact  as  the  basis  of  an  inductive  argument 
to  locate  other  nodes  at  which  "continue"  is  the  optimal  decision. 
Consider  states  (u,s,e)  for  which  we  know  (by  inductive  hypothesis) 
"continue"  to  be  the  optimal  decision  at  all  later  decision 
points.  Other  decision  points  which  might  be  encountered  after 
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(u,s,e)  are  (i,s,e)  and  (i,s+l,e),  for  i=u-l , u-2 , . . . , 1 .  We  com¬ 
pare  expected  costs  to  determine  conditions  under  which  preemption 
at  (u,s,e)  is  desirable: 

u+s  s 

Lcont  =  tu+s+1)  (8-e)  +  J  i [pe+ (1-p) 8] +  l  i(8-e), 

i=s+l  i=l 

u+s+1  s+1 

Lore  =  (u+s+1) C+  l  i  [pe+ (1-p) 8]  +  l  i(8-e). 

i=s+2  i=l 

Thus  preemption  is  optimal  if  and  only  if 

(u+s+1) C  +  (u+s+1) Ipe+ (1-p) 8]  +  (s+1) (8-e)  < 

(u+s+1)  (8-e)  +  (s+1) Ips+ Cl-p) 8] / 

or  C  <  -jl^Ipe-Cl+plE]  . 

Moving  system  state  variables  to  the  left  yields 

<  i[pB-U+p)e]  -  1.  (3) 

Note  that  inequality  (3)  specializes  to  inequality  (2)  when  u=l. 
Again,  parameters  C,  p,  e ,  and  8  determine  a  threshold, 

(1/C)  [p8-(l+p)s]  -  1.  Preemption  is  desirable  (assuming  no  later 
preemption)  if  and  only  if  the  ratio  (s+l)/u  is  less  than  the 
threshold. 

In  Figure  8,  decision  points  of  the  state  transition  dia¬ 
gram  are  replaced  by  their  (s+l)/u  ratio.  For  which  states  will 
inequality  (2)  hold?  The  value  (1/C) [p8~(l+p)e]  -  1  determines  a 
line  in  Figure  8  such  that  the  inequality  holds  for  all  decision 
points  to  the  left  of  the  line  and  for  none  to  the  right  of  the 
line.  The  line  passes  through  the  relative  location  of  the  imag¬ 
inary  node  (0,-l,e)  for  all  parameter  values.  If  p8~(l+p)£  ^  C, 
then  (1/C)  [p8- (1+p)  e] -1<0  and  all  decision  points  lie  to  the  right 
of  the  line.  If  C=0,  then  Cl/C)  £ p 8—  (1+p) c]  -  1  is  infinite  and 
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Figure  8. — Critical  ratios  of  system  states. 
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all  decision  points  lie  to  the  left  of  the  line. 

Results  derived  so  far  are  sufficient  to  prove  the  optimality 
of  the  decision  "continue"  at  all  decision  points  to  the  right  of 
the  line.  For  u=l,  we  obtain  the  result  directly  (see  inequality 
(2)).  For  u>l,  we  must  observe  that  from  any  decision  point, 
(u,s,e),  all  later  decision  points  have  fewer  unstarted  requests. 
Thus  by  induction  on  the  number  of  remaining  unstarted  requests, 
it  can  be  shown  that  "continue"  is  the  optimal  decision  at  every 
decision  point  (u,s,e)  for  which  (s+l)/u  >  (1/C) [pB-(l+p)e]  -  1. 
Next,  we  must  consider  decision  points  for  which  (s+l)/u  < 

(1/C) [p$-(l+p)e]  -  1.  We  divide  these  into  the  following  three 
groups  for  consideration: 

l  s+1 

(I)  those  for  which  ^-IpB- (1+p)  e] -1  <  yzy. 


s+1 


s+2 


(II)  those  for  which  ^y  <  ^[pf3-  (1+p)  e] -1  <  yry 


s+2  1 

and  (III)  those  for  which  y^y  <  ^Ipg- (1+p) e] -1 . 

For  group  I,  all  later  decision  points  are  known  to  have  "continue" 
as  an  optimal  strategy.  Therefore,  the  earlier  analysis  which  led 
to  inequality  (3)  applies  here  to  establish  "preempt"  as  the 
optimal  decision. 

Group  II  decision  points  require  a  more  extensive  analysis. 
For  any  such  point  we  may  determine  a  unique  integer,  j,  such  that 


S+1  ro  li  ,  .  ,  s+1 

y=j-  <  [pB-  (1+p)  e]-l  *  u-j'-T' 

We  will  prove  by  induction  on  j  that,  if  "preempt"  is  optimal 
from  (u-l,s,e),  (u-2 , s , e)  ,  .  .  .  ,  (u-j,s,e)  and  "continue"  is  optimal 


from  all  other  decision  points  reachable  from  (u,s,s) ,  then 
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"preempt"  is  the  optimal  decision  at  (u,s,e). 

We  will  calculate  the  difference  in  expected  cost  between 

preempting  and  not  preempting  at  (u,s,e)  and  taking  known  optimal 

decisions  thereafter.  The  uncertainty  of  the  situation  lies  in 

unstarted  requests.  We  classify  the  possible  eventual  outcomes 

by  the  number  of  unstarted  requests  initiated  which  finish  in  s 

units  before  the  first  one  which  does  not.  With  probability 
k-1 

77 =  p  (1-p)  ,  exactly  k-1  unstarted  requests  will  turn  out  to 
be  of  service  time  e  before  one  turns  out  to  be  of  service  time 
3.  If  the  decision  at  (u,s,e)  is  "preempt",  then  all  later  deci¬ 
sions  will  be  "continue"  since  Cl/C) [p8~ Cl+p) e ] -1  <  (s+2)/(u-l). 

If  the  decision  at  (u,s,s)  is  "continue",  and  if  the  kth  unstarted 
request  is  the  first  one  which  turns  out  to  be  long,  where  k<j, 
then  it  will  be  preempted,  and  the  difference  in  expected  cost 
between  "preempt"  and  "continue"  at  (u,s,e)  is  calculated  as 
follows.  With  "preempt"  at  (u,s,s)  rather  than  "continue",  the 
first  k-1  completions  occur  (8-£)-(e+C)  =  (8-2e-C)  earlier,  the 
kth  completion  occurs  (e+C)  later,  and  all  other  completion  times 
are  unchanged.  Thus  if  the  kth  unstarted  request  is  the  first 
long  one,  the  change  in  cost  of  preempting  at  (u,s,s)  rather  than 
not  preempting  is  AC  =  (e+C) - Ck-1)  (8-2e-C) , 

K 

=  kC- (k-1) 8+ (2k- 1) e . 

If  the  first  j  unstarted  requests  all  are  short,  then  preempting 
at  (u,s,e)  allows  the  u  unstarted  requests  to  finish  (8-e-C) 
earlier  while  the  s  already  preempted  requests  are  delayed  by  C, 
and  the  request  preempted  at  (u,s,e)  is  delayed  by  an  expected 
amount  of  C+ je+ (u- j) Ipe+ (1-p) 8]  •  The  change  in  cost  with  pre- 
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emption  at  (u,s,e)  if  the  first  j  unstarted  requests  are  short  is 


AC> j  =  C+je  +  (u- j)  [pe+ (1-p)  g]  +  sC  -  u(g-e-C) 

=  (u+s+1) C  -  j  (g-2e )  -  (u-j)  [pg- (1+p) e]  . 

The  overall  change  in  expected  cost  with  preemption  at  (u,s,e)  is 
given  by 


Le  -  L  =  j  it.  AC,  + 

S-pre  S-cont  k“x  k  k 


1  - 


TT7 


k=l 


AC>j, 


=  l  pk-1(  1-p)  [kC-  Ck-1)  6+  (2k-l)  e] 
k=l 

+  p][(u+s+l)C  -  j  (g-2e)  -  (u-j)  [pg- (1+p) e]]  , 

=  j  pk_1 [kC-  (k-1) g  +  (2k-l) e]  -  jj  pk [kC- Ck-1) g+ (2k-l) e] 
k=l  k=l 


:v 

=  C+e  +  l 
k=l 


+  jp^  (C-g+2e)  +  p3 [(u+s+l-j) C- Cu-j)  [pg—  (1+p) e]]  , 

■] 


pk[ (k+l)C-kg+ (2k-l) e]  -  pk [kC- (k-1) 3+ (2k-l) e ] 


-  p-1  [  jC-  Cj-1)  3+  C2  j-1)  e]  +  jp^  (C-g  +  2e) 
+  p-^  (u+s+1- j  )  C  -  (u-j )  [pg- (1+p) e]  , 


j-1  , 

=  (C+e)  +  l  pK (C-g+2e)  -  pD(g-e) 
k=l 

+  pi [ (u+s+1- j) C  -  (u- j)  [pg- (1+p) e]] , 


i  + 

c  - 

j;x  k  j 

Z  P  +  P 

b  +  i  +  2'ly1pk  +  Pj 

1 

1 — 1 

II 

i 

k=l 

i — 1 

i  n 

Thus 

LS-pre  “  LS-cont 


V  *1 

2  P 

r~ 

ll 

o 

i _ 

+  p  pu+s+l-j)C  -  (u-j)  [pg- (1+p)  e] 


C  -  [pg- (1+p) e] 

+  p3  [Cu+s+l-j)  C  -  Cu-j)  [pg-  (1+p)  e]]. 


Remembering  that 


fry  <  i[p(S-(l+p)e]  -  1, 
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we  know  that 

(u+s+1- j ) C  <  (u- j)  [p3- (1+p) e] 

and 

C  <  ^^zyt-l^C  <  [p3-Cl+p)e]. 

Thus  both  terms  of  the  change  in  expected  cost  are  negative,  and 
it  is  demonstrated  that  a  decision  of  "preempt"  at  Cu,s,£)  re¬ 
duces  expected  cost. 

By  induction  on  j,  the  optimality  of  "preempt"  at  each 
group  II  decision  point  can  be  proven.  For  j=l,  "preempt"  is 
known  optimal  at  Cu-l,s,£)  because  it  is  a  group  I  decision  point. 
For  j>l,  "preempt"  is  known  optimal  at  Cu-j,s,e:)  because  it  is  a 
group  I  decision  point,  and  at  Cu-l,s,e),  (u-2 , s , e)  ,  .  . . ,  and 
(u-j+l,s,e)  because  they  are  group  II  decision  points  with  fewer 
unstarted  requests. 

Finally,  we  consider  group  III  decision  points.  Each  of 
these,  (u,s,e),  is  such  that  (u-l,s+l,e)  and  Cu-1,s,e)  are  either 
group  I  or  II  decision  points,  or  group  III  decision  points  with 
fewer  unstarted  requests.  Thus  again,  by  induction,  "preempt" 
can  be  proven  optimal  at  all  group  III  decision  points  if  we  can 
prove  the  following:  If  Cs+ j  +  1) / (u- j }  <  Cl/C)  [pB- Cl+p) e] -1  and 
the  optimal  path  from  Cu-l,s,e)  involves  at  most  j  preemptions, 
then  the  optimal  decision  at  Cu,s,e)  is  "preempt." 

The  proof  is  by  contradiction.  We  assume  that  "continue" 
is  the  optimal  decision  from  (u,s,e).  Then  an  optimal  schedule, 
S-cont,  from  (u,s,e)  may  have  the  following  character: 

1.  Finish  the  request  currently  assigned. 
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2.  Finish  the  u  previously  unstarted  requests  using  as 
many  as  j  preemptions . 

3.  Finish  the  s  already  preempted  requests. 

But  compare  this  with  an  alternative,  S-pre: 

1.  Preempt  the  request  currently  assigned. 

2.  Finish  the  u  unstarted  requests  in  an  order  and 
manner  equivalent  to  that  under  the  other  schedule. 

3.  Finish  the  request  preempted  at  (u,s,s) . 

4.  Finish  the  s  already  preempted  requests. 

Under  the  schedule  using  preemption  initially,  u  requests 
finish  (3-e-C)  sooner,  s  requests  finish  C  later  and  the  expected 
amount  that  the  other  request  is  delayed  is  at  most 

C  +  jC  +  ulpe+  Cl-p)  0]  . 

Therefore  the  change  in  expected  cost  of  using  this  alternative 
schedule  rather  than  the  one  presumed  optimal  is  such  that 

LS-pre"  LS-cont  <  sC+c+3c+u [pe+ ( 1-p) 6 ] -u (3-e-C) , 

LS-pre-  LS-cont  <  (u+s+l+j) C-u[p8- (1+p) e] . 

Remembering  that  0  <  (s+ j+1) / (u- j )  <  Cl/C) [p$- Cl+p) £] -1 ,  we  find 
that  (u+s+l)C  <  (u-j)  [p3-(l+p)e]  and  also  C  <  [p(3- (1+p)  e]  . 
Multiplying  the  latter  inequality  by  j  and  adding  the  former  yields 

(u+s+l+j) C  <  u [pg- (1+p) e] . 

Therefore 

LS-pre“  LS-cont  <  (u+s+l+j ) C-uIpg- (1+p) e]  <  0. 

Because  expected  cost  is  reduced  by  preempting  at  a  decision  point 
(u,s,e),  "preempt"  is  the  optimal  decision  at  (u,s,e). 

For  j=l,2,...  we  apply  this  argument  to  prove  "preempt" 
optimal  at  group  III  decision  points  for  which 
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2±i+i  <  i[p[3-(l+p)e]-l 


<  S±}±2. 

U-J-l 


From  such  a  point,  at  most  j  preemptions  will  be  used  in  any 
optimal  schedule  since  j  preemptions  lead  to  a  decision  point 
from  which  "continue"  is  optimal  thereafter.  Inductively, 
"preempt"  is  guaranteed  optimal  for  group  III  decision  points  with 
successively  larger  values  of  j. 

This  rather  involved  proof  has  shown  that  in  state  (u,s,e), 
preemption  is  optimal  if  and  only  if 

(s+l)/u  <  Cl/C)  [pB  —  Cl+p)  e] -1  • 

The  significance  of  this  is  that  we  see  that  preemption  decisions 
should  not  be  based  only  on  the  request  currently  assigned  and  one 
alternative  request,  but  must  be  made  in  light  of  the  entire 
system  load.  As  more  requests  are  preempted,  the  desirability  of 
additional  preemptions  declines  because  the  ratio  of  requests 
delayed  by  the  preemption  to  requests  with  potential  of  bene- 
fitting  from  the  preemption  increases. 

Only  very  specialized  request  classes  might  have  service 
time  distributions  which  resemble  the  spike  function  discussed 
here.  However,  situations  do  arise  in  which  the  expected  re¬ 
maining  service  time  of  a  request  increases  at  certain  points  of 
attained  service.  Consider  a  class  of  student  FORTRAN  jobs.  A 
certain  proportion  of  the  jobs  contain  syntactic  errors  and  are 
terminated  after  the  compilation  phase.  Those  that  compile  suc¬ 
cessfully  might  be  expected  to  do  a  significant  amount  of  compu¬ 
tation  before  terminating.  Thus,  knowing  the  expected  compilation 
time,  we  may  pick  a  threshold  of  attained  service  beyond  which 
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a  job  is  likely  to  run  a  significant  time  longer.  The  preceding 
discussion  indicates  that  in  scheduling  for  such  a  request  class , 
it  might  be  desirable  to  preempt  requests  which  have  apparently 
compiled  successfully  in  order  to  attempt  other  compilations. 
However,  as  the  backlog  of  preempted  requests  increases,  the 
desirability  of  additional  preemption  decreases. 

Optimal  preemption's  relation  to  rank 

A  more  intuitive  justification  of  the  result  proved  above 
can  be  obtained  by  relating  to  and  extending  the  idea  of  rank. 
When  preemption  was  free,  the  optimal  strategy  was  to  assign  at 
every  instant  the  request  of  current  smallest  rank.  With  sig¬ 
nificant  preemption  costs,  however,  we  must  weigh  the  costs  of 
delaying  everyone  by  a  preemption  against  allowing  the  job  cur¬ 
rently  assigned  to  continue  although  it  is  not  of  smallest  rank. 
In  the  system  we  have  treated,  all  service  times  are  drawn  from 
a  two-spike  distribution  function.  In  the  interesting  case  it  is 
always  more  desirable  to  assign  an  unstarted  request  than  one 
which  did  not  complete  in  its  first  e  units  of  service  and  is 
known  to  be  of  longer  variety.  Thus  if  there  are  u  unstarted 

requests  in  system,  s  requests  which  have  been  preempted  after 

e  units  of  service,  and  one  request  currently  assigned  which  has 
just  failed  to  complete  in  e  units  of  service,  we  should  preempt 
only  if  the  cost  incurred  in  preemption  is  less  than  the  cost  of 
allowing  the  request  currently  assigned  to  complete  out  of 
smallest  rank  order.  The  cost  of  preemption  is  (u+s+l)C  since 
each  job  in  system  is  delayed  by  C  units.  The  cost  of  running 

the  request  currently  assigned  out  of  smallest  rank  order  is 
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determined  as  follows.  The  remainder  of  the  request  currently- 
assigned  is  tied  for  lowest  priority  in  the  system  according  to 
smallest  rank  order.  By  completing  it  first,  all  (u+s)  other 
requests  in  system  are  delayed  by  $“£  while  its  completion  is 
moved  up  by  the  expected  time  of  completing  the  other  requests 
which  is  u[p£  + (1-p) 6]+s  (3-e) .  The  expected  change  in  cost  of 
completing  the  assigned  request  is 

[ (u+s)  ($-e) ] - [u(pe+ (1-p) 6) +s  (B-e) ]  =  u [p3- (1+p) e] • 
Balancing  this  cost  against  that  of  preemption,  we  conclude,  as 
before,  that  preemption  is  desirable  if  and  only  if 

(u+s+l)C  <  u  [ p 6  —  (1+p)  e  ]  , 

or  S±i  <  i[pB-(l+p)e]  -  1. 


Arbitrary  service  time  distributions 

In  considering  more  complex  service  time  distribution  func¬ 
tions,  another  difficulty  arises.  The  rank  function  as  defined 
earlier  is  no  longer  sufficient  to  select  the  appropriate  request 
for  processor  assignment.  For  example,  consider  a  system  with 
two  request  classes: 


and 


F1(t)  = 


F2(t)  = 


0  0<t<£ 

P  £<t<3 

i  e<t 
0  0<t<a 

1  a<t . 


Assume  that  g>£(l+p)/p  and  (e/p) <a<p£+ Cl-p) 6 .  Then  the  current 
definition  of  rank  indicates  R  ( j ^ , 0 )  =  (£/p)  and  tf(j2,0)=a.  Yet 
with  enough  requests  in  system,  preemption  could  be  prohibitively 
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expensive ,  and  the  effective  commitment  in  selecting  a  class  1 
request  is  pe+(l-p)g.  Thus  we  hope  that  the  definition  of  rank 
can  be  extended  to  take  preemption  costs  into  account  and  indicate 
the  preferability  of  assigning  a  class  2  request  with  a  suffi¬ 
ciently  heavy  system  load.  A  step  in  this  direction  is  a  change 
in  the  definition  of  expected  investment  to  include  the  commit¬ 
ment  to  a  preemption  when  the  request  fails  to  complete  in  its 
rank  quantum.  We  define 

e  °° 

Jc  Cj  ,t,e) =/  x9xGi(x,t)+(e+NC)  /  S^Cx,^ 

where  N  =  number  of  requests  in  system. 

This  definition  includes  the  fact  that  with  the  probability  that 
j  will  not  finish  in  an  assigned  quantum  of  e,  N  requests  will 
be  delayed  C  time  units  while  j  is  preempted. 

Increasing  rank  distributions 

In  Chapter  III,  we  found  that  the  exponential  distribution 
of  service  times  was  the  unique  distribution  of  constant  rank. 

Thus  under  free  preemption  and  identical  linear  losses,  scheduling 
is  arbitrary  in  a  system  whose  single  request  class  has  exponen¬ 
tially  distributed  service  times.  However,  as  soon  as  preemption 
costs  are  significant,  no  preemption  is  desirable  in  such  a 
situation . 

Another  implication  of  the  results  of  Chapter  III  was  that 
optimal  scheduling  when  the  single  request  class  has  a  service 
time  distribution  with  increasing  rank  closely  approximates  the 
preemptive  processor  sharing  discipline.  (SR  +  PPS  as  basic 
allocation  unit  0.)  Preemption  costs  greater  than  zero  add 
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significant  complexity  to  the  scheduling  problem  in  this  situation 
because  the  cost  of  preemption  must  be  weighted  against  the  de¬ 
sirability  of  assigning  the  request  with  least  attained  service. 

A  useful  distinction  at  this  point  is  between  distributions 
of  finite  and  of  infinite  mean.  In  attempting  to  approximate  ser¬ 
vice  time  distributions,  only  distributions  of  finite  mean  are  of 
interest.  This  means  that  although  expected  remaining  service 
time  may  increase  with  attained  service,  it  cannot  increase  with¬ 
out  bound.  Thus  in  order  to  have  a  finite  mean,  expected  remain¬ 
ing  service  time  must  asymptotically  approach  some  bound,  b,  as 
attained  service  increases.  This  indicates  that  any  increasing 
rank  distribution  of  finite  mean  will  be  such  that  the  remaining 
service  time  distribution  with  sufficiently  great  attained  service 
approaches  an  exponential  distribution  with  parameter  1/b  where 
b  is  the  upper  bound  on  expected  remaining  service  time.  For  any 
preemption  cost,  C>0,  there  is  some  attained  service  at  which  the 
remaining  service  time  distribution  is  so  close  to  exponential 
that  preemption  is  no  longer  worthwhile.  This  fact  is  acknowl¬ 
edged  in  some  proposed  scheduling  rules.  In  attempting  to  limit 
the  frequency  of  preemption,  Coffman  suggests  modifications  of  RR 
and  FBn  which  place  requests  in  a  background  FIFO  queue  once  they 
have  attained  a  certain  amount  of  service  without  completing  [12] . 

Considerations  of  practicality  require  that  service  times 
of  a  particular  request  class  be  bounded  by  some  finite  maximum 
value.  In  this  situation,  attained  service  sufficiently  close 
to  the  maximum  value  guarantees  that  the  remaining  service  time 
distribution  is  of  decreasing  rank  and  no  preemption  beyond  this 
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point  is  desirable.  It  can  be  shown  that  truncating  an  exponen¬ 
tial  distribution  at  M, 


0<t<M 


fit) 


t>M , 


produces  a  decreasing  rank  distribution  so  that  even  free  pre¬ 
emption  is  non-optimal .  Thus,  any  distribution  which  is  a  rea¬ 
sonable  approximation  of  the  distribution  of  service  times  for 
some  request  class  has  a  value  of  attained  service  beyond  which 
preemption  is  no  longer  desirable. 

A  Hyperexponential  distribution 

We  will  consider  now  in  greater  detail  a  system  whose  single 
request  class  has  a  hyperexponential  distribution  with  parameters 
ot ,  A^  and 


f  (t)  =  1-ae  ^^-(l-cOe  ^2t  where  A^  <  A^« 


If  we  have  two  unstarted  requests  in  a  system,  then  for  reasons 
discussed  below,  the  optimal  scheduling  strategy  to  follow  until 
one  of  them  is  completed  has  the  following  character: 


1.  Assign  for  q-^  units. 

2.  Assign  j  ^  for  q-]_+q2  URits  • 

3.  Assign  for  q2+<33  units. 


2k+l .  Assign  for  <32k+<^2k+l  un:*-ts  • 


Assign  j2  for  q2k+i+<32k+2 


2k+2 . 


units . 
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The  length  of  each  interval  depends  on  several  factors.  Intu¬ 
itively,  it  is  the  amount  by  which  the  two  requests  may  differ  in 
attained  service  before  their  difference  in  rank  is  sufficiently 
great  to  outweigh  the  cost  incurred  by  a  preemption.  Because  rank 
increases  more  rapidly  for  small  values  of  attained  service  Cthis 
is  true  because  it  approaches  an  upper  bound  asymptotically) ,  we 
would  expect  that  <3i<(3i+i  •  That  is,  the  intervals  of  assign¬ 

ment  become  longer  and  longer  as  rank  changes  more  and  more 
slowly.  Finally,  as  seen  earlier,  if  C>0,  we  reach  a  point  of 
attained  service  at  which  rank  is  so  close  to  its  upper  bound 
that  the  cost  of  preemption  can  never  be  justified  even  if  the 
attained  service  of  the  other  request  becomes  arbitrarily  large. 
Thus,  for  some  i,  q^=°°.  As  C  is  made  larger,  all  q^  become 
larger.  As  C^2_^l^/^2  becomes  smaller  (that  is,  the  distribution 
becomes  nearly  exponential) ,  the  q^  become  larger. 

Let  Xq  denote  the  least  value  of  attained  service  such  that 
preemption  in  favor  of  a  request  with  attained  service  Xq  is  never 
desirable.  We  may  develop  an  expression  for  x^  as  follows. 

Assume  that  two  requests  have  attained  service  times  x  and  y  re¬ 
spectively  where  y>x  and  that  the  request  with  attained  service  y 
is  assigned  to  the  processor.  Then  the  expected  costs  of  com¬ 
pleting  the  two  requests  with  and  without  an  immediate  preemption 
are : 

with  preemption 


oo 


CO 


LS-pre  =  2C  +  2/  s  9qG(s,x)  +  /  s  3  G(s,y), 
^  0  0S 


and  without  preemption 


00 


00 


LS-cont 


=  2 /  s  3  G(s,y)  +  /  s  3sG(s,x). 
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Then  is  the  smallest  value  of  x  such  that  for  arbitrarily 


large  y. 


2C  +  2 /  s  9  G(s,x)  +  /  s  9  G(s,y) 


>  2/  s  9  G(s,y)  +  /  s  9  G(s,x), 


or 


2C  +  /  s  9  G(s,x) 
0  s 


CO 

>  /  s  9  G  (s,y)  . 

0 


Substituting  in  the  known  expression  for  expected  remaining 
service  time  for  an  hyperexponential  distribution,  letting 
A =  n  and  ^  =  y*  and  letting  y  -*■  °°  yields 
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therefore 


X0  ~  y 
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-2aC 

-a) (y-n-2Cny) 

2aCny 


-a)  (y-n-2Cny ) \  . 
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If  X  =  1/15,  X2  =  1/10,  a  =  .5,  and  C  =  1, 


then 


Xq  =  30  In 


=  30  In  (1 . 5 )  ~  12.5. 


But  12.5  is  the  mean  of  this  distribution.  Thus,  for  this  example, 
preemption  should  never  be  done  in  favor  of  a  request  whose  at¬ 
tained  service  is  greater  than  the  mean  service  time.  With  more 
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than  two  requests  in  system,  the  threshold  beyond  which  a  request 
should  not  be  preempted  for  is  even  lower. 

In  order  to  determine  how  large  C  must  be  in  order  to  make 
preemption  never  desirable,  we  set  Xq=  0  and  solve  for  C.  This 
yields  the  conclusion  that  preemption  is  never  desirable  if 
C  >  ( 1 — ot )  ( X 2 — A ^ )  / 2X  2  • 

Implications  for  traditional  rules 

In  this  chapter,  we  have  seen  that  the  desirability  of  pre¬ 
emption  is  dependent  on  the  number  of  requests  in  system  and  the 
properties  of  the  service  time  distribution  function.  Only  a  few 
scheduling  rules  recognize  the  decreased  desirability  of  pre¬ 
emption  under  heavy  load.  The  input-dependent  rules  treated  by 
Coffman  do  [12] .  Greenberger  [26]  mentions  the  possibility  of  a 
round-robin  scheme  in  which  the  quanta  become  larger  under  heavy 
load.  McKinney  and  Patterson  [44]  found  a  significant  performance 
improvement  resulting  from  the  use  of  state  dependent  quanta. 

When  the  request  load  of  a  system  is  variable  and  unpredictable, 
the  use  of  a  state-dependent  scheduling  rule  allows  adaptation  to 
the  current  environment. 

If  the  distribution  of  service  times  has  a  hyperexponential 
form,  and  if  we  do  not  have  sufficient  information  to  partition 
the  results  into  two  or  more  request  classes  with  distinctive 
service  time  distributions,  then,  as  we  have  seen,  two  goals  of 
scheduling  should  be  Cl)  serve  first  the  request  with  the  least 
attained  service  time  (smallest  rank) ,  and  (2)  successive  quanta 
assigned  a  single  request  should  be  longer  and  longer  since  the 
rate  at  which  rank  increases  is  decreasing.  A  scheduling  rule 
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with  precisely  these  properties  is  the  CTSS  scheduling  algorithm 
[52] .  It  is  an  FBn  rule  in  which  the  quantum  received  at  each 
queue  level  is  twice  that  received  at  the  previous  level.  This 
rule  differs  from  an  optimal  rule  for  a  hyperexponential  distri¬ 
bution  only  in  that  its  quanta  lengths  increase  only  geometrically, 
while  we  have  seen  that  the  optimal  rule  must  assign  an  infinite 
quantum  once  a  particular  threshold  of  attained  service  is 
reached . 


CHAPTER  VI 


OTHER  CONSIDERATIONS 

Scheduling  under  non-linear  losses 

In  Chapters  III  through  V,  the  decline  in  utility  of  a  par¬ 
ticular  computational  result  was  presumed  to  be  strictly  propor¬ 
tional  to  the  time  from  its  arrival  to  its  completion.  However, 
as  was  seen  in  Chapter  I,  the  assumption  of  linear  losses  is 
justifiable  only  when  trying  to  characterize  a  group  of  loss 
functions.  An  individual  loss  function  is  typically  of  a  form 
too  complex  to  be  communicated  efficiently  to  the  system,  even  in 
the  rare  case  when  its  exact  form  is  known  to  the  user. 

Many  objective  functions  for  scheduling  which  are  not  based 
explicitly  on  the  concept  of  cost  or  loss  may  be  expressed  in 
terms  of  loss  functions  of  various  forms.  The  goals  of  minimiz¬ 
ing  average  time  in  system,  minimizing  average  waiting  time, 
minimizing  average  completion  time,  minimizing  average  number  in 
system,  and  minimizing  average  lateness  (where  the  lateness  of  a 
request  is  the  difference  between  its  completion  time  and  its 
preset  due  time)  are  all  equivalent  to  minimizing  cost  while 
assuming  identical  linear  loss  functions.  When  weights  are 
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assigned  to  request  classes  reflecting  the  relative  importance  of 
good  service  to  each  class,  minimizing  weighted  time  in  system, 
weighted  waiting  time,  weighted  number  in  system,  and  weighted 
lateness  are  each  equivalent  to  minimizing  cost  under  linear 
losses  where  the  rate  of  loss  for  each  class  is  proportional  to  the 
weight  associated  with  that  class. 

Other  measures  require  more  complex  loss  functions.  Maxi¬ 
mizing  earliness  (max[0,due  time  -  completion  time])  and  minimiz¬ 
ing  tardiness  (max[0 , completion  time  -  due  time])  are  both  equiv¬ 
alent  to  minimizing  total  loss  with  the  bounded  linear  loss  func¬ 
tions  described  in  Chapter  I.  When  maximizing  earliness,  the 
sloping  portion  of  the  loss  function  extends  from  the  time  of 
arrival  until  the  due  time.  For  minimizing  tardiness,  the  slop¬ 
ing  portion  starts  at  the  due  time  and  continues  indefinitely. 

While  consideration  of  non-linear  loss  functions  makes  the 
determination  of  generally  optimal  rules  extremely  difficult,  it 
is  possible  to  anticipate  the  general  influences  of  various  loss 
function  forms  on  the  optimal  rules  for  linear  loss  situations. 

If  losses  tend  to  be  convex,  so  that  the  rate  of  loss  increases 
with  time  in  system,  then  less  preemptions  are  motivated.  With 
the  oldest  job  in  system  having  the  highest  current  rate  of  loss, 
it  is  less  advisable  to  preempt  in  favor  of  a  more  recent  arrival 
than  if  losses  are  linear.  Similarly,  if  loss  functions  are  con¬ 
cave,  the  greatest  savings  are  made  by  completing  requests  before 
they  get  very  far  into  their  initial  period  of  a  rather  high  rate 
of  loss.  A  request  which  somehow  has  remained  in  the  system  for 
a  while  has  a  relatively  low  current  rate  of  loss,  and  is  not  of 
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pressing  concern.  Thus  preemption  in  favor  of  a  new  arrival  is 
motivated  more  frequently  than  under  linear  losses. 

We  have  seen  that  both  distribution  functions  and  loss 
functions  tend  to  motivate  anywhere  from  an  oldest  first  (FIFO) 
to  a  newest  first  (PPS)  form  of  priority.  When  considered  in  com 
bination,  these  influences  may  be  either  conflicting  or  re-enforc 
ing  as  is  seen  in  Table  3.  When  the  influences  are  conflicting, 
a  compromise  must  be  made.  Round-robin  is  a  frequently  used  com¬ 
promise  rule.  Table  3  presumes  preemption  cost  to  be  zero.  Sig¬ 
nificant  preemption  cost  forces  PPS  to  become  FBR,  and  generally 
discourages  preemption. 

When  loss  functions  fail  to  have  a  monotone  second  deriva¬ 
tive,  selection  rules  are  not  sufficient  to  provide  near  optimal 
scheduling.  The  particular  characteristics  of  the  loss  function 
associated  with  each  request  must  be  considered. 

Round- robin:  the  compromise  rule 

It  has  already  been  pointed  out  that  the  round- robin  rules, 
RR  and  PS,  are  conspicuously  absent  from  the  rules  to  which  SR, 
the  optimal  rule  under  linear  losses,  specializes.  Only  in  the 
very  special  case  of  exponentially  distributed  service  times  was 
RR  found  to  minimize  expected  cost,  and  in  that  situation  all 
rules  yield  identical  expected  cost.  Here  we  widen  the  claim  to 
encompass  all  forms  of  loss  functions.  We  state  the  claim  as 
follows.  If  arrival  times  are  not  deterministic  and  an  arrival 
may  occur  during  the  processing  of  other  requests,  then  no  combi¬ 
nation  of  request  classes,  each  defined  by  an  arrival  process,  a 
service  time  distribution,  and  a  loss  function,  creates  a 
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TABLE  3 

INFLUENCE  OF  LOSS  FUNCTIONS  AND 
SERVICE  TIME  DISTRIBUTIONS  ON  SCHEDULING 


Service 

Time  Distribution 

Decreasing 

Constant 

Increasing 

Rank 

Rank 

Rank 

Shape 

Convex 

FIFO 

FIFO 

compromise 

necessitated 

of 

Loss 

Linear 

FIFO 

all  schedules 
yield 

equal  costs 

PPS 

Functions 

Concave 

compromise 

necessitated 

PPS 

PPS 

Ill 


situation  in  which  round-robin  or  processor-sharing  scheduling  is 
optimal  unless  FIFO,  a  simpler  rule,  is  also  optimal.  This  claim 
is  difficult  to  prove,  but  we  present  an  intuitive  justification. 

A  new  arrival  deserves  identical  service  to  a  request  with  some 
positive  attained  service  only  if  their  outlooks  (remaining  ser¬ 
vice  time  distributions  and  marginal  loss  functions)  are  iden¬ 
tical.  This  occurs  only  in  situations  of  linear  loss  and  expo¬ 
nential  or  geometric  service  time  distribution  where  FIFO  does  as 
well  as  any  other  rule. 

Yet,  round- robin  has  proven  empirically  to  be  a  satisfac¬ 
tory  scheduling  rule  for  processor  allocation  in  many  actual 
computer  systems.  This  is  true  because,  as  was  suggested  earlier, 
it  is  indeed  a  good  compromise  rule.  While  another  rule  may  be 
better  than  RR  in  a  particular  situation,  RR  is  likely  to  be  much 
less  sensitive  to  slight  perturbations  in  the  situation.  Thus 
when  the  situation  is  not  perfectly  static  or  when  loss  rates  and 
service  time  distributions  are  known  imprecisely,  round- robin  is 
likely  to  be  more  stable  in  providing  acceptable  performance  than 
another  rule  whose  performance  is  critically  dependent  on  precise 
information. 

Another  intuitive  justification  of  round-robin  scheduling 
arises  when  the  system  has  a  dynamic  rather  than  fixed  view  of 
the  loss  function  associated  with  each  request.  Up  to  now,  we 
have  associated  a  fixed  loss  function  with  each  request  purely  on 
the  basis  of  its  class  identity.  Alternatively,  within  a  request 
class,  requests  indistinguishable  upon  arrival  might  actually 
have  different  loss  functions  correlated  with  their  own  actual 
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service  times.  Consider  the  possibility  of  two  requests,  i  and  j, 
from  the  same  request  class,  and  thus  indistinguishable  upon 
arrival,  having  distinct  loss  functions,  1^ Ct)  and  1 j (t) .  Assume 
that  their  actual  service  times  are  t^  and  tj  respectively,  and 
that  their  loss  functions  are  related  by  l^Ckt^)  =  lj  (kt \/k>0. 
Thus  the  current  loss  rates  of  the  two  requests  when  their  times 
in  system  are  equal  multiples  of  their  actual  service  times  are 
inversely  proportional  to  their  actual  service  times. 

If  the  system  acknowledges  that  the  loss  functions  of  any 
two  requests  may  depend  on  their  actual  service  times  as  well  as 
their  class  identity,  then  it  may  use  the  attained  service  times 
of  the  requests  to  calculate  their  current  "estimated"  loss  func¬ 
tion.  If,  within  a  request  class,  the  loss  functions  of  any  two 
requests  are  related  as  described  above,  than  the  "estimated" 
loss  function  becomes  flatter  and  flatter  with  increasing  attained 
service.  This  dynamic  view  of  loss  functions  motivates  the  prac¬ 
tice  of  giving  each  request  a  share  of  every  interval  as  a  pre¬ 
caution  against  having  a  request  ignored  whose  actual  service 
time  is  short  so  that  its  actual  loss  rate  is  higher  than  the 
system  estimates. 

The  ease  of  implementation  and  relatively  low  overhead  of 
scheduling  under  RR  are  other  motivations  for  its  use. 

The  Implementation  of  SR 

While  we  have  found  SR  to  be  an  optimal  scheduling  rule  in 
certain  situations  with  respect  to  minimizing  expected  cost,  its 
practical  utility  is  restricted  by  some  assumptions  of  the  model. 
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First,  it  was  shown  in  Chapter  V  that  when  the  overhead  of  preemp¬ 
tion  is  significant,  SR  is  no  longer  optimal.  Second,  no  consid¬ 
eration  has  been  given  to  the  overhead  of  scheduling.  The  SR  rule 
requires  more  information  and  more  computation  than  do  other 
rules.  This  extra  scheduling  overhead  must  be  weighed  against  the 
gain  in  goodness  of  scheduling  in  order  to  judge  SR's  practical 
worth . 

While  SR  has  never  been  implemented  fully,  we  present  here 
a  plan  for  an  implementation  which  will  keep  scheduling  overhead 
manageable.  First,  a  record  of  all  requests  during  some  period 
is  used.  The  record  must  show  all  information  known  upon  the 
arrival  of  a  request  along  with  the  actual  service  time  of  the 
request.  Then  we  partition  the  requests  into  request  classes  ac¬ 
cording  to  their  a  priori  attributes  with  the  following  goals: 

1.  Make  the  distributions  for  various  request  classes 
distinctly  different. 

2.  Make  the  distributions  be  of  decreasing  rank  or  at 
least  have  only  a  few  points  of  increasing  rank. 

3.  Create  few  enough  classes  that  the  volume  of  information 
to  be  maintained  is  manageable  and  that  each  distribu¬ 
tion  is  based  on  enough  sample  points  to  have  statisti¬ 
cal  significance. 

From  the  resulting  service  time  distributions,  we  may  create 
tables  which  indicate  a  quantum  of  assignment  as  a  function  of 
request  class  and  attained  service  of  the  request  to  be  assigned 
and  the  current  rank  of  the  next  lowest  rank  job  in  system.  The 
quantum  is  the  length  of  time  that  the  request  of  current  smallest 
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rank  can  be  run  before  its  rank  is  greater  than  that  of  the  re¬ 
quest  of  current  second  smallest  rank. 

When  the  three  goals  of  request  class  creation  are  ade¬ 
quately  fulfilled,  the  tables  indicating  quanta  of  assignment 
remain  of  manageable  size.  Scheduling  than  consists  of  maintain¬ 
ing  a  single  queue  of  requests  ordered  by  increasing  current 
rank.  Each  queue  entry  indicates  the  class  identity,  the  attain¬ 
ed  service  time,  and  the  current  rank  of  the  request.  At  each 
completion,  the  job  of  current  lowest  rank  is  assigned  for  a 
quantum  determined  by  table  look-up.  If  the  quantum  elapses 
before  the  request  is  completed,  the  uncompleted  request  with  its 
new  attained  service  and  new  rank  is  ordered  in  the  request 
queue,  and  the  new  job  of  current  lowest  rank  is  assigned,  again 
for  a  quantum  determined  by  table  look-up.  At  each  arrival,  the 
rank  of  the  arriving  request  determines  the  position  at  which  it 
will  enter  the  queue.  If  its  rank  is  lower  than  that  of  the 
currently  assigned  request,  a  preemption  is  required.  If  its 
rank  is  greater  than  that  of  the  currently  assigned  request  but 
lower  than  that  of  any  waiting  request,  then  no  preemption  is 
done,  but  a  new  quantum  look-up  is  needed  in  order  to  determine 
the  largest  quantum  for  which  the  request  currently  assigned  can 
be  continued  without  its  rank  increasing  above  that  of  the  new 
arrival . 

When  preemption  costs  are  significant,  the  tables  may  be 
constructed  to  require  that  the  rank  of  an  assigned  request  be 
some  fixed  amount  greater  than  that  of  another  request  before  a 


preemption  is  performed. 
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It  seems  apparent  that  the  scheduling  overhead  of  an  imple¬ 
mentation  of  SR  can  be  held  to  a  reasonable  level.  But  the  more 
important  question  remains:  Are  the  gains  made  by  SR  scheduling 
over  other  scheduling  rules  of  sufficient  magnitude  to  warrant 
incursion  of  even  this  small  increase  in  scheduling  overhead? 

To  answer  this  question  extensive  empirical  study  with  real  ser¬ 
vice  time  distributions  will  be  required. 

Some  related  scheduling  problems 

MODEL o /  as  described  in  Chapter  I,  is  based  on  such  general 
assumptions  that  no  research  has  yielded  an  optimal  scheduling 
rule  without  restrictions  to  limit  some  dimensions  of  its  gener¬ 
ality.  Yet  it  is  not  difficult  to  determine  some  characteristics 
in  which  MODELQ  should  be  made  still  more  general  in  order  to 
encompass  most  of  the  scheduling  problems  arising  in  real  computer 
systems . 

First,  a  current  computer  system  is  seldom  allowed  the 
luxury  of  allocating  all  its  resources  as  a  single  package.  The 
added  complexity  of  allocating  two  or  more  resources  simultaneous¬ 
ly  is  great  when  there  is  significant  dependency  in  need  of  the 
resources  by  the  users. 

In  a  computer  system,  the  resources  to  be  allocated  are  of 
two  basic  types;  memory  devices  and  processor  devices.  At  least 
one  of  each  is  needed  to  perform  any  useful  function  since  pro¬ 
cessor  type  devices  (CPU,s,  data  channels,  disk  drives,  etc.)  only 
accomplish  work  by  manipulating  memory  type  devices  (arithmetic 
registers,  core  memory,  tape  reel,  etc.).  Few  attempts  at  solving 
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multiple  resource  allocation  have  appeared  in  the  literature. 

Two  of  the  more  successful  [9,  42]  both  use  only  time  averages  of 
resource  needs  rather  than  a  more  precise  representation  of  the 
simultaneity  of  need  for  memory  and  processor. 

In  MODEL Q  we  have  considered  only  systems  with  homogeneity 
of  utility  function  types.  In  reality  some  requests  have  real¬ 
time  constraints  on  their  completion,  while  others  have  associated 
utility  functions  of  much  different  forms.  In  computer  operating 
systems,  better  mechanisms  should  be  available  for  the  user  to 
communicate  a  description  of  his  estimated  utility  function  to 
the  system.  The  current  practice  of  allowing  specification  of 
only  a  single  parameter  which  is  taken  to  be  relative  priority 
restricts  the  amount  of  information  conveyed.  Only  if  the 
charging  algorithm,  scheduling  algorithm,  and  utility  function 
communication  mechanism  are  made  to  cooperate,  can  the  system 
maximize  the  service  it  provides  the  user  community.  Further, 
with  this  cooperation,  the  system  may  use  economic  pressures  to 
encourage  and  discourage  various  styles  of  system  usage  [47] . 

Such  manipulation  of  the  user  is  desirable  because  aggregate 
individual  choice  does  not  necessarily  lead  to  maximum  total  ser¬ 
vice  rendered  13] . 

Finally,  when  dealing  with  multiple  resources,  the  cost  of 
preemption  may  no  longer  be  treated  as  simply  the  loss  of  time  on 
a  single  resource.  Preempting  a  request  from  one  resource  is 
likely  to  have  implications  on  the  allocation  of  other  resources 
due  to  the  simultaneity  constraints  of  resource  usage. 
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Summary  of  significant  results 

When  service  times  are  non-deterministic ,  knowing  expected 
service  time  is  not  sufficient  to  minimize  expected  total  loss  to 
users  due  to  delay.  Full  knowledge  of  the  service  time  distri¬ 
bution  of  each  request  class  can  lead  to  a  significant  reduction 
in  the  expected  total  loss  by  employing  the  general  scheduling 
principle  of  minimizing  the  ratio  of  expected  processor  time 
invested  to  likelihood  of  completion.  The  scheduling  rule, 
"Smallest  Rank",  which  is  based  on  this  principle,  minimizes 
expected  total  loss  for  a  finite  set  of  initially  available 
requests  when  loss  functions  are  linear  and  preemption  is  free. 

Preemption,  even  when  its  cost  is  negligible,  should  be 
employed  only  at  arrivals  of  new  requests,  and  at  points  of 
attained  service  of  the  request  currently  assigned  where  its  rank 
increases.  This  result  is  compatible  with  various  previous 
results  stated  in  terms  of  other  dynamic  characteristics  of  ser¬ 
vice  time  distributions  such  as  mean  residual  life  and  failure 
rate . 

Arbitrary  arrival  patterns,  even  when  successive  inter¬ 
arrival  times  are  independent  selections  from  a  fixed  distribution, 
can  make  an  optimal  scheduling  rule  extremely  difficult  to  find. 
However,  when  losses  are  linear,  the  existence  of  a  Poisson  arri¬ 
val  process  does  not  seem  to  disturb  the  optimality  of  selection 
rules  known  optimal  under  no  arrivals.  The  rules  SRPT/c  and  SR 
seem  to  minimize  total  expected  rate  of  loss  under  Poisson  arri¬ 
vals  when  service  times  are  known  precisely  and  probabilistically, 
respectively . 
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When  the  amount  of  processor  time  lost  in  a  preemption  is 
significant,  the  preemption  decision  is  dependent  on  the  current 
state  of  the  system.  The  amount  saved  by  allowing  certain  higher 
priority  requests  to  go  ahead  of  the  request  currently  in  service 
must  be  balanced  against  the  cost  of  delaying  every  request  in 
system  by  the  amount  of  time  taken  in  preemption. 

If  the  overhead  of  scheduling  itself  is  presumed  negligible, 
any  service  time  distribution  motivates  a  scheduling  strategy  in 
which  every  request  class  has  a  threshold  of  attained  service  at 
which  requests  should  be  placed  in  a  lower  priority  background 
queue  which  receives  service  in  a  strictly  first-come,  first- 
served  fashion.  In  particular,  many  RR  and  FBn  with  identical 
quanta  strategies  could  be  improved  by  the  addition  of  a  back¬ 
ground  FIFO  queue,  unless  the  overhead  of  the  management  of  a 
background  queue  is  prohibitive. 

Among  the  goals  of  this  research,  has  been  the  advancement 
of  the  cooperative  use  of  techniques  from  traditionally  distinct 
areas  of  endeavor.  The  scheduling  problems  in  modern  multipro- 
grammed  computer  systems  contain  elements  requiring  tools  from 
both  production  scheduling  and  queueing  theory.  This  author 
anticipates  both  an  increased  need  for  and  an  increased  realiza¬ 
tion  of  a  merging  of  these  areas. 
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LIMITING  VALUES  OF  INTERVAL  RANK 
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Because  g^(x,t)  has  at  most  a  finite  number 
its  limit  exists  as  £+0 ,  although  it  may  be 
number,  x,  making  i?(j,t,£)  equal  to  00 , 


xGi  (x ,  t) 
(x,t) 


of  discontinuities, 
0,  00 ,  or  a  finite 
,  or  1/x. 


APPENDIX  II 


EXAMPLES  OF  SCHEDULE  TREES 

Figures  9,  10,  and  11  on  the  following  pages  show  three 
schedule  trees  for  completing  requests  a,  b,  and  c  of  class  i 
where  =  3  is  the  maximum  number  of  time  units  needed  by  any 
request  of  class  i.  Figure  12  shows  the  processor  assignment 
functions  which  result  from  use  of  each  of  the  schedule  trees 
with  a  particular  set  of  actual  service  times.  In  the  schedule 
trees,  the  right  branch  from  each  node  represents  the  no-comple- 
tion  path,  while  the  left  branch  represents  the  completion  path. 
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Figure  9.--A  FIFO  schedule  tree. 


(a, 2) 


Figure  10. — A  weird  schedule  tree. 
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(a,  1) 


Figure  11. — A  round-robin  schedule  tree 
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If  actual  service  times  are 
t  =3,^=2,  and  tc=l  then  the 
processor  assignment  functions 
of  various  schedule  trees  are: 


RR 

FIFO 

WEIRD 

t=0  1  2  3  4  5  6 


a 
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c 

a 
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»»;>-* 
i  •  <  *  i 
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i  i 

i  i 
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i  ' 
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1  1 

a 

c 

b 

a 
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Figure  12. — The  use  of  schedule  trees 
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